2024-08-17T02:32:41.619 INFO:root:teuthology version: 1.2.0 2024-08-17T02:32:41.642 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-17T02:32:41.726 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288 branch: wip-mchangir-mgr-volumes-print-thread-name-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} email: mchangir@redhat.com first_in_suite: false job_id: '7858288' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-mchangir-mgr-volumes-print-thread-name-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - filesystem is online with fewer MDS than max_mds - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 2577e8139c6b8786f9d6d6ea21b8da9244b9260e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: enable_coprs: - ceph/el9 flavor: default sha1: 2577e8139c6b8786f9d6d6ea21b8da9244b9260e kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-mgr-volumes-print-thread-name-quincy sha1: 2577e8139c6b8786f9d6d6ea21b8da9244b9260e owner: scheduled_mchangir@teuthology priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5995 sha1: 2577e8139c6b8786f9d6d6ea21b8da9244b9260e sleep_before_teardown: 0 suite: fs suite_branch: wip-mchangir-mgr-volumes-print-thread-name-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 2577e8139c6b8786f9d6d6ea21b8da9244b9260e targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMyjE/ovyyv8jkDimXUkeH3RLshdrtDmzbzTklszNSDNsX64CwAH69KFULc/On5kSIoI/8n7k24YFhLqaxnqvTY= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKo7qihD/ULkIowbEL69Jvq8IIu6heU6oXFnxNctDMbd3NEHXgkMVIA+k0bltnB+bLXxq+Kf6C142VdpMlWJ/Q4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-17_01:10:27 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-17T02:32:41.726 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa; will attempt to use it 2024-08-17T02:32:41.726 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks 2024-08-17T02:32:41.727 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-17T02:32:41.731 INFO:teuthology.task.internal:Checking packages... 2024-08-17T02:32:41.752 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2577e8139c6b8786f9d6d6ea21b8da9244b9260e' 2024-08-17T02:32:41.752 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-17T02:32:41.752 INFO:teuthology.packaging:ref: None 2024-08-17T02:32:41.752 INFO:teuthology.packaging:tag: None 2024-08-17T02:32:41.752 INFO:teuthology.packaging:branch: wip-mchangir-mgr-volumes-print-thread-name-quincy 2024-08-17T02:32:41.752 INFO:teuthology.packaging:sha1: 2577e8139c6b8786f9d6d6ea21b8da9244b9260e 2024-08-17T02:32:41.752 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-mchangir-mgr-volumes-print-thread-name-quincy 2024-08-17T02:32:41.903 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1375.g2577e813 2024-08-17T02:32:41.904 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-17T02:32:41.911 INFO:teuthology.task.internal:no buildpackages task found 2024-08-17T02:32:41.911 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-17T02:32:41.918 INFO:teuthology.task.internal:Saving configuration 2024-08-17T02:32:41.933 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-17T02:32:41.940 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-17T02:32:41.963 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-17 02:27:56.920197', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMyjE/ovyyv8jkDimXUkeH3RLshdrtDmzbzTklszNSDNsX64CwAH69KFULc/On5kSIoI/8n7k24YFhLqaxnqvTY='} 2024-08-17T02:32:41.980 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-17 02:27:56.921821', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKo7qihD/ULkIowbEL69Jvq8IIu6heU6oXFnxNctDMbd3NEHXgkMVIA+k0bltnB+bLXxq+Kf6C142VdpMlWJ/Q4='} 2024-08-17T02:32:41.980 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-17T02:32:41.986 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-08-17T02:32:41.987 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-08-17T02:32:41.987 INFO:teuthology.run_tasks:Running task console_log... 2024-08-17T02:32:42.048 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7eff7ea9bac0>, signals=[15]) 2024-08-17T02:32:42.048 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-17T02:32:42.054 INFO:teuthology.task.internal:Opening connections... 2024-08-17T02:32:42.054 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:32:42.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:32:42.132 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:32:42.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:32:42.207 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-17T02:32:42.214 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-08-17T02:32:42.232 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-08-17T02:32:42.232 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-08-17T02:32:42.290 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-08-17T02:32:42.290 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-08-17T02:32:42.291 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-08-17T02:32:42.291 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-08-17T02:32:42.291 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-08-17T02:32:42.291 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-08-17T02:32:42.291 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-17T02:32:42.291 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-08-17T02:32:42.291 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-08-17T02:32:42.291 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-17T02:32:42.291 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-08-17T02:32:42.291 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-08-17T02:32:42.291 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-17T02:32:42.291 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-17T02:32:42.292 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-08-17T02:32:42.317 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-08-17T02:32:42.334 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-08-17T02:32:42.334 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-08-17T02:32:42.391 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-08-17T02:32:42.391 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-08-17T02:32:42.391 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-08-17T02:32:42.391 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-08-17T02:32:42.391 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-08-17T02:32:42.391 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-08-17T02:32:42.391 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-17T02:32:42.391 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-08-17T02:32:42.391 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-08-17T02:32:42.391 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-17T02:32:42.391 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-08-17T02:32:42.392 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-08-17T02:32:42.392 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-17T02:32:42.392 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-17T02:32:42.392 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-08-17T02:32:42.417 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-17T02:32:42.425 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-17T02:32:42.484 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-17T02:32:42.484 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-08-17T02:32:42.487 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-08-17T02:32:42.503 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-17T02:32:42.508 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-17T02:32:42.508 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-08-17T02:32:42.545 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-08-17T02:32:42.600 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-17T02:32:42.650 INFO:teuthology.run_tasks:Running task kernel... 2024-08-17T02:32:42.664 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-08-17T02:32:42.664 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-08-17T02:32:42.664 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-08-17T02:32:42.664 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-17T02:32:42.665 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-17T02:32:42.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:32:42.681 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-08-17T02:32:42.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:32:42.684 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-08-17T02:32:42.843 DEBUG:teuthology.task.kernel:sha1 for client.1 is 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 2024-08-17T02:32:42.843 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "7503b862ff5bc4e556b3daee1c7881aa3e90ed62"... 2024-08-17T02:32:42.843 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-08-17T02:32:42.847 DEBUG:teuthology.task.kernel:sha1 for client.0 is 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 2024-08-17T02:32:42.847 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "7503b862ff5bc4e556b3daee1c7881aa3e90ed62"... 2024-08-17T02:32:42.847 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-08-17T02:32:42.860 INFO:teuthology.orchestra.run.smithi142.stdout:5.14.0-437.el9.x86_64 2024-08-17T02:32:42.860 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 2024-08-17T02:32:42.860 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-08-17T02:32:42.861 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum upgrade -y linux-firmware 2024-08-17T02:32:42.865 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-437.el9.x86_64 2024-08-17T02:32:42.865 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 2024-08-17T02:32:42.865 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-08-17T02:32:42.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum upgrade -y linux-firmware 2024-08-17T02:32:45.767 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.2 MB 00:01 2024-08-17T02:32:46.418 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.2 MB 00:01 2024-08-17T02:32:47.515 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-08-17T02:32:48.361 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-08-17T02:32:52.625 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.6 MB 00:01 2024-08-17T02:32:53.511 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.6 MB 00:01 2024-08-17T02:32:54.300 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 18 kB 00:00 2024-08-17T02:32:55.212 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 84 kB/s | 18 kB 00:00 2024-08-17T02:32:55.421 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 23 MB/s | 22 MB 00:00 2024-08-17T02:32:56.166 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 27 MB/s | 22 MB 00:00 2024-08-17T02:33:00.977 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-08-17T02:33:01.555 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-08-17T02:33:03.126 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-08-17T02:33:03.142 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-08-17T02:33:03.142 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-08-17T02:33:03.142 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-08-17T02:33:03.142 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-08-17T02:33:03.142 INFO:teuthology.orchestra.run.smithi016.stdout: linux-firmware noarch 20240716-145.el9 baseos 383 M 2024-08-17T02:33:03.142 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:33:03.142 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-08-17T02:33:03.142 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-08-17T02:33:03.142 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 1 Package 2024-08-17T02:33:03.143 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:33:03.143 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 383 M 2024-08-17T02:33:03.143 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-08-17T02:33:03.646 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-08-17T02:33:03.657 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-08-17T02:33:03.658 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-08-17T02:33:03.658 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-08-17T02:33:03.658 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-08-17T02:33:03.658 INFO:teuthology.orchestra.run.smithi142.stdout: linux-firmware noarch 20240716-145.el9 baseos 383 M 2024-08-17T02:33:03.658 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:33:03.658 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-08-17T02:33:03.658 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-08-17T02:33:03.658 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 1 Package 2024-08-17T02:33:03.658 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:33:03.659 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 383 M 2024-08-17T02:33:03.659 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-08-17T02:33:08.856 INFO:teuthology.orchestra.run.smithi142.stdout:linux-firmware-20240716-145.el9.noarch.rpm 74 MB/s | 383 MB 00:05 2024-08-17T02:33:08.856 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-08-17T02:33:08.856 INFO:teuthology.orchestra.run.smithi142.stdout:Total 74 MB/s | 383 MB 00:05 2024-08-17T02:33:09.445 INFO:teuthology.orchestra.run.smithi016.stdout:linux-firmware-20240716-145.el9.noarch.rpm 64 MB/s | 383 MB 00:06 2024-08-17T02:33:09.445 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-08-17T02:33:09.445 INFO:teuthology.orchestra.run.smithi016.stdout:Total 61 MB/s | 383 MB 00:06 2024-08-17T02:33:11.947 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-08-17T02:33:12.034 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-08-17T02:33:12.034 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-08-17T02:33:12.677 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-08-17T02:33:12.766 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-08-17T02:33:12.766 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-08-17T02:33:15.325 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-08-17T02:33:15.325 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-08-17T02:33:16.159 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-08-17T02:33:16.160 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-08-17T02:33:24.930 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-08-17T02:33:25.353 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : linux-firmware-20240716-145.el9.noarch 1/2 2024-08-17T02:33:25.434 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : linux-firmware-20240219-143.el9.noarch 2/2 2024-08-17T02:33:25.893 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-08-17T02:33:26.274 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : linux-firmware-20240716-145.el9.noarch 1/2 2024-08-17T02:33:26.349 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: linux-firmware-20240219-143.el9.noarch 2/2 2024-08-17T02:33:26.349 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 1/2 2024-08-17T02:33:26.355 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : linux-firmware-20240219-143.el9.noarch 2/2 2024-08-17T02:33:27.054 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 2/2 2024-08-17T02:33:27.054 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:33:27.054 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-08-17T02:33:27.054 INFO:teuthology.orchestra.run.smithi142.stdout: linux-firmware-20240716-145.el9.noarch 2024-08-17T02:33:27.054 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:33:27.054 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-08-17T02:33:27.124 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: linux-firmware-20240219-143.el9.noarch 2/2 2024-08-17T02:33:27.124 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 1/2 2024-08-17T02:33:27.382 INFO:teuthology.task.kernel:Downloading kernel 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 on client.1... 2024-08-17T02:33:27.382 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7503b862ff5bc4e556b3daee1c7881aa3e90ed62 2024-08-17T02:33:27.525 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/7503b862ff5bc4e556b3daee1c7881aa3e90ed62/centos/9/flavors/default/x86_64/ 2024-08-17T02:33:27.526 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc3_g7503b862ff5b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/7503b862ff5bc4e556b3daee1c7881aa3e90ed62/centos/9/flavors/default/x86_64/ --input-file=- 2024-08-17T02:33:27.528 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi142.front.sepia.ceph.com to complete... 2024-08-17T02:33:27.874 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 2/2 2024-08-17T02:33:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:33:27.875 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-08-17T02:33:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: linux-firmware-20240716-145.el9.noarch 2024-08-17T02:33:27.875 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:33:27.875 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-08-17T02:33:28.188 INFO:teuthology.task.kernel:Downloading kernel 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 on client.0... 2024-08-17T02:33:28.189 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7503b862ff5bc4e556b3daee1c7881aa3e90ed62 2024-08-17T02:33:28.334 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/7503b862ff5bc4e556b3daee1c7881aa3e90ed62/centos/9/flavors/default/x86_64/ 2024-08-17T02:33:28.334 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc3_g7503b862ff5b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/7503b862ff5bc4e556b3daee1c7881aa3e90ed62/centos/9/flavors/default/x86_64/ --input-file=- 2024-08-17T02:33:28.336 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi016.front.sepia.ceph.com to complete... 2024-08-17T02:33:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17 02:33:42 URL:https://2.chacra.ceph.com/r/kernel/testing/7503b862ff5bc4e556b3daee1c7881aa3e90ed62/centos/9/flavors/default/x86_64/kernel-6.11.0_rc3_g7503b862ff5b-1.x86_64.rpm [360536130/360536130] -> "/tmp/kernel.x86_64.rpm" [1] 2024-08-17T02:33:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:FINISHED --2024-08-17 02:33:42-- 2024-08-17T02:33:42.463 INFO:teuthology.orchestra.run.smithi142.stderr:Total wall clock time: 15s 2024-08-17T02:33:42.464 INFO:teuthology.orchestra.run.smithi142.stderr:Downloaded: 1 files, 344M in 15s (23.5 MB/s) 2024-08-17T02:33:42.467 INFO:teuthology.task.kernel:Installing kernel 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 on client.1... 2024-08-17T02:33:42.467 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-08-17T02:33:44.543 INFO:teuthology.orchestra.run.smithi142.stdout:Verifying... ######################################## 2024-08-17T02:33:44.693 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing... ######################################## 2024-08-17T02:33:44.744 INFO:teuthology.orchestra.run.smithi142.stdout:Updating / installing... 2024-08-17T02:33:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17 02:33:47 URL:https://2.chacra.ceph.com/r/kernel/testing/7503b862ff5bc4e556b3daee1c7881aa3e90ed62/centos/9/flavors/default/x86_64/kernel-6.11.0_rc3_g7503b862ff5b-1.x86_64.rpm [360536130/360536130] -> "/tmp/kernel.x86_64.rpm" [1] 2024-08-17T02:33:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:FINISHED --2024-08-17 02:33:47-- 2024-08-17T02:33:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:Total wall clock time: 19s 2024-08-17T02:33:47.475 INFO:teuthology.orchestra.run.smithi016.stderr:Downloaded: 1 files, 344M in 19s (18.4 MB/s) 2024-08-17T02:33:47.479 INFO:teuthology.task.kernel:Installing kernel 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 on client.0... 2024-08-17T02:33:47.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-08-17T02:33:49.688 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying... ######################################## 2024-08-17T02:33:49.752 INFO:teuthology.orchestra.run.smithi016.stdout:Preparing... ######################################## 2024-08-17T02:33:49.805 INFO:teuthology.orchestra.run.smithi016.stdout:Updating / installing... 2024-08-17T02:33:52.824 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-6.11.0_rc3_g7503b862ff5b-1 ######################################## 2024-08-17T02:33:58.288 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-6.11.0_rc3_g7503b862ff5b-1 ######################################## 2024-08-17T02:34:32.116 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi142.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-08-17T02:34:32.117 DEBUG:teuthology.orchestra.run.smithi142:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-08-17T02:34:32.196 INFO:teuthology.orchestra.run.smithi142.stdout:/boot/System.map-6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:32.196 INFO:teuthology.orchestra.run.smithi142.stdout:/boot/config-6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/boot/vmlinuz-6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/System.map 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/config 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-08-17T02:34:32.197 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-08-17T02:34:32.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-08-17T02:34:32.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-08-17T02:34:32.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/crc32c-intel.ko 2024-08-17T02:34:32.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-08-17T02:34:32.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-08-17T02:34:32.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-08-17T02:34:32.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-08-17T02:34:32.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/serpent-avx2.ko 2024-08-17T02:34:32.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-08-17T02:34:32.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-08-17T02:34:32.198 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kernel 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kernel/cpu 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kernel/cpu/mce 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kvm 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kvm/kvm-amd.ko 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kvm/kvm-intel.ko 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kvm/kvm.ko 2024-08-17T02:34:32.199 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto 2024-08-17T02:34:32.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/ansi_cprng.ko 2024-08-17T02:34:32.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/anubis.ko 2024-08-17T02:34:32.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/arc4.ko 2024-08-17T02:34:32.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx 2024-08-17T02:34:32.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_memcpy.ko 2024-08-17T02:34:32.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_pq.ko 2024-08-17T02:34:32.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_raid6_recov.ko 2024-08-17T02:34:32.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_tx.ko 2024-08-17T02:34:32.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_xor.ko 2024-08-17T02:34:32.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/raid6test.ko 2024-08-17T02:34:32.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/authenc.ko 2024-08-17T02:34:32.200 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/authencesn.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/blake2b_generic.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/blowfish_common.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/blowfish_generic.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/camellia_generic.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cast5_generic.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cast6_generic.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cast_common.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/ccm.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cmac.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/crc32_generic.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/crypto_engine.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/crypto_user.ko 2024-08-17T02:34:32.201 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cts.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/des_generic.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/dh_generic.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/ecc.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/ecdh_generic.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/echainiv.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/essiv.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/fcrypt.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/khazad.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/lz4.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/lz4hc.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/md4.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/michael_mic.ko 2024-08-17T02:34:32.202 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/pcbc.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/pcrypt.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/rmd160.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/seed.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/serpent_generic.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/tcrypt.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/tea.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/twofish_common.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/twofish_generic.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/vmac.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/wp512.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/xcbc.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/xor.ko 2024-08-17T02:34:32.203 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/xxhash_generic.ko 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/acpi_ipmi.ko 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/acpi_pad.ko 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/ec_sys.ko 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/platform_profile.ko 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/sbs.ko 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/sbshc.ko 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/video.ko 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/acard-ahci.ko 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/ahci_platform.ko 2024-08-17T02:34:32.204 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/ata_generic.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/libahci_platform.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_acpi.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_ali.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_amd.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_artop.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_atiixp.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_atp867x.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_cmd640.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_cmd64x.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_cypress.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_efar.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_hpt366.ko 2024-08-17T02:34:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_hpt37x.ko 2024-08-17T02:34:32.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_hpt3x2n.ko 2024-08-17T02:34:32.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_hpt3x3.ko 2024-08-17T02:34:32.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_it8213.ko 2024-08-17T02:34:32.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_it821x.ko 2024-08-17T02:34:32.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_jmicron.ko 2024-08-17T02:34:32.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_marvell.ko 2024-08-17T02:34:32.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_mpiix.ko 2024-08-17T02:34:32.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_netcell.ko 2024-08-17T02:34:32.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_ninja32.ko 2024-08-17T02:34:32.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_ns87410.ko 2024-08-17T02:34:32.206 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_ns87415.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_oldpiix.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_opti.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_optidma.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_pcmcia.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_pdc2027x.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_piccolo.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_rdc.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_sch.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_serverworks.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_sil680.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_sis.ko 2024-08-17T02:34:32.207 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_sl82c105.ko 2024-08-17T02:34:32.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_triflex.ko 2024-08-17T02:34:32.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_via.ko 2024-08-17T02:34:32.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pdc_adma.ko 2024-08-17T02:34:32.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_inic162x.ko 2024-08-17T02:34:32.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_mv.ko 2024-08-17T02:34:32.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_nv.ko 2024-08-17T02:34:32.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_promise.ko 2024-08-17T02:34:32.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_qstor.ko 2024-08-17T02:34:32.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_sil.ko 2024-08-17T02:34:32.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_sil24.ko 2024-08-17T02:34:32.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_sis.ko 2024-08-17T02:34:32.208 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_svw.ko 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_sx4.ko 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_uli.ko 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_via.ko 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_vsc.ko 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/auxdisplay 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/auxdisplay/ks0108.ko 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/base 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/base/regmap 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/base/regmap/regmap-i2c.ko 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/base/regmap/regmap-mmio.ko 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/bcma 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/bcma/bcma.ko 2024-08-17T02:34:32.209 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block 2024-08-17T02:34:32.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/aoe 2024-08-17T02:34:32.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/aoe/aoe.ko 2024-08-17T02:34:32.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/brd.ko 2024-08-17T02:34:32.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/drbd 2024-08-17T02:34:32.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/drbd/drbd.ko 2024-08-17T02:34:32.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/floppy.ko 2024-08-17T02:34:32.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/loop.ko 2024-08-17T02:34:32.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/mtip32xx 2024-08-17T02:34:32.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-08-17T02:34:32.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/nbd.ko 2024-08-17T02:34:32.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/null_blk 2024-08-17T02:34:32.210 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/null_blk/null_blk.ko 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/pktcdvd.ko 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/rbd.ko 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/virtio_blk.ko 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/xen-blkback 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/xen-blkfront.ko 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/zram 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/zram/zram.ko 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hangcheck-timer.ko 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/amd-rng.ko 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/intel-rng.ko 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-08-17T02:34:32.211 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/via-rng.ko 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/virtio-rng.ko 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_si.ko 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/lp.ko 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/mwave 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/mwave/mwave.ko 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ppdev.ko 2024-08-17T02:34:32.212 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tlclk.ko 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm.ko 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_atmel.ko 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_infineon.ko 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_nsc.ko 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_tis.ko 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/uv_mmtimer.ko 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/virtio_console.ko 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/p4-clockmod.ko 2024-08-17T02:34:32.213 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/powernow-k8.ko 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/speedstep-lib.ko 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/ccp 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/ccp/ccp.ko 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat/qat_common 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-08-17T02:34:32.214 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/padlock-aes.ko 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/padlock-sha.ko 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/virtio 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dca 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dca/dca.ko 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/devfreq 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma/dw 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma/dw/dw_dmac.ko 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma/ioat 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma/ioat/ioatdma.ko 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac 2024-08-17T02:34:32.215 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/amd64_edac.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/e752x_edac.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/edac_mce_amd.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i3000_edac.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i3200_edac.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i5100_edac.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i5400_edac.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i7300_edac.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i7core_edac.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i82975x_edac.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/ie31200_edac.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/sb_edac.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/x38_edac.ko 2024-08-17T02:34:32.216 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/firmware 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/firmware/edd.ko 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/firmware/iscsi_ibft.ko 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/gpio 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/gpio/gpio-ich.ko 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/gpio/gpio-viperboard.ko 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-appleir.ko 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-aureal.ko 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-axff.ko 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-dr.ko 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-elecom.ko 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-elo.ko 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-emsff.ko 2024-08-17T02:34:32.217 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-gaff.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-gt683r.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-gyration.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-holtek-kbd.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-holtek-mouse.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-holtekff.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-hyperv.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-icade.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-keytouch.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-kye.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-lcpower.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-led.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-lenovo.ko 2024-08-17T02:34:32.218 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-logitech-dj.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-multitouch.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-ortek.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-penmount.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-petalynx.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-picolcd.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-pl.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-plantronics.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-primax.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-rmi.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-arvo.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-common.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-isku.ko 2024-08-17T02:34:32.219 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-kone.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-konepure.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-lua.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-pyra.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-ryos.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-savu.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-saitek.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-samsung.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-sensor-hub.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-sjoy.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-sony.ko 2024-08-17T02:34:32.220 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-speedlink.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-steelseries.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-sunplus.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-thrustmaster.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-tivo.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-tmff.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-topseed.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-twinhan.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-uclogic.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-waltop.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-wiimote.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-xinmo.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-zpff.ko 2024-08-17T02:34:32.221 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-zydacron.ko 2024-08-17T02:34:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/uhid.ko 2024-08-17T02:34:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/wacom.ko 2024-08-17T02:34:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hv 2024-08-17T02:34:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hv/hv_balloon.ko 2024-08-17T02:34:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hv/hv_utils.ko 2024-08-17T02:34:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hv/hv_vmbus.ko 2024-08-17T02:34:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon 2024-08-17T02:34:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/abituguru.ko 2024-08-17T02:34:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/abituguru3.ko 2024-08-17T02:34:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/acpi_power_meter.ko 2024-08-17T02:34:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ad7414.ko 2024-08-17T02:34:32.222 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ad7418.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adc128d818.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm1025.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm1026.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm1029.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm1031.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm9240.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ads7828.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7410.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7411.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7462.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7470.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7475.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7x10.ko 2024-08-17T02:34:32.223 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/amc6821.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/applesmc.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/asb100.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/asc7621.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/asus_atk0110.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/atxp1.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/coretemp.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/dme1737.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ds1621.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ds620.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/emc1403.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/emc6w201.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/f71805f.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/f71882fg.ko 2024-08-17T02:34:32.224 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/f75375s.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/fam15h_power.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/fschmd.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/g760a.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/g762.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/gl518sm.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/gl520sm.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/hwmon-vid.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/i5500_temp.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/i5k_amb.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ibmaem.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ibmpex.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ina209.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ina2xx.ko 2024-08-17T02:34:32.225 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/it87.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/k10temp.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/k8temp.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lineage-pem.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm63.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm73.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm75.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm77.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm78.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm80.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm83.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm85.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm87.ko 2024-08-17T02:34:32.226 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm90.ko 2024-08-17T02:34:32.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm92.ko 2024-08-17T02:34:32.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm93.ko 2024-08-17T02:34:32.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm95234.ko 2024-08-17T02:34:32.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm95241.ko 2024-08-17T02:34:32.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm95245.ko 2024-08-17T02:34:32.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc2945.ko 2024-08-17T02:34:32.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4151.ko 2024-08-17T02:34:32.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4215.ko 2024-08-17T02:34:32.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4222.ko 2024-08-17T02:34:32.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4245.ko 2024-08-17T02:34:32.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4260.ko 2024-08-17T02:34:32.227 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4261.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max16065.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max1619.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max1668.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max197.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max6639.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max6650.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max6697.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/mcp3021.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/nct6683.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/nct6775-core.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/nct6775.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/nct7802.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pc87360.ko 2024-08-17T02:34:32.228 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pc87427.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pcf8591.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/max16064.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/max34440.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/max8688.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-08-17T02:34:32.229 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/powr1220.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sch5627.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sch5636.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sch56xx-common.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sht15.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sht21.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/shtc1.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sis5595.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/smsc47b397.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/smsc47m1.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/smsc47m192.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/thmc50.ko 2024-08-17T02:34:32.230 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/tmp102.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/tmp103.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/tmp401.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/tmp421.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/via-cputemp.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/via686a.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/vt1211.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/vt8231.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83627ehf.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83627hf.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83781d.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83791d.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83792d.ko 2024-08-17T02:34:32.231 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83793.ko 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83795.ko 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83l785ts.ko 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83l786ng.ko 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/algos 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-08-17T02:34:32.232 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-i801.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-isch.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-parport.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-08-17T02:34:32.233 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-08-17T02:34:32.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-08-17T02:34:32.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-08-17T02:34:32.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-via.ko 2024-08-17T02:34:32.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-08-17T02:34:32.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-08-17T02:34:32.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-dev.ko 2024-08-17T02:34:32.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-mux.ko 2024-08-17T02:34:32.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-08-17T02:34:32.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-smbus.ko 2024-08-17T02:34:32.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-stub.ko 2024-08-17T02:34:32.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband 2024-08-17T02:34:32.234 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/ib_cm.ko 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/ib_core.ko 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/ib_umad.ko 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/iw_cm.ko 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/rdma_cm.ko 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/cxgb4 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mlx4 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mlx5 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-08-17T02:34:32.235 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mthca 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/ipoib 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/iser 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/isert 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/srp 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/srpt 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/gameport 2024-08-17T02:34:32.236 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/gameport/fm801-gp.ko 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/gameport/gameport.ko 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/joydev.ko 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/keyboard 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/keyboard/gpio_keys.ko 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/appletouch.ko 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/bcm5974.ko 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/cyapatp.ko 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/elan_i2c.ko 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/sermouse.ko 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-08-17T02:34:32.237 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/synaptics_usb.ko 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/vsxxxaa.ko 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/rmi4 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/rmi4/rmi_core.ko 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio/altera_ps2.ko 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio/arc_ps2.ko 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio/serio_raw.ko 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/sparse-keymap.ko 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/leds-blinkm.ko 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/leds-lm3530.ko 2024-08-17T02:34:32.238 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/leds-lp3944.ko 2024-08-17T02:34:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/leds-ss4200.ko 2024-08-17T02:34:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger 2024-08-17T02:34:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-08-17T02:34:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-08-17T02:34:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-08-17T02:34:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-08-17T02:34:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-08-17T02:34:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-08-17T02:34:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-08-17T02:34:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-08-17T02:34:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md 2024-08-17T02:34:32.239 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/bcache 2024-08-17T02:34:32.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/bcache/bcache.ko 2024-08-17T02:34:32.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-bio-prison.ko 2024-08-17T02:34:32.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-cache-smq.ko 2024-08-17T02:34:32.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-cache.ko 2024-08-17T02:34:32.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-crypt.ko 2024-08-17T02:34:32.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-delay.ko 2024-08-17T02:34:32.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-flakey.ko 2024-08-17T02:34:32.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-log-userspace.ko 2024-08-17T02:34:32.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-multipath.ko 2024-08-17T02:34:32.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-queue-length.ko 2024-08-17T02:34:32.240 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-raid.ko 2024-08-17T02:34:32.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-round-robin.ko 2024-08-17T02:34:32.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-service-time.ko 2024-08-17T02:34:32.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-switch.ko 2024-08-17T02:34:32.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-thin-pool.ko 2024-08-17T02:34:32.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-verity.ko 2024-08-17T02:34:32.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/persistent-data 2024-08-17T02:34:32.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-08-17T02:34:32.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/raid0.ko 2024-08-17T02:34:32.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/raid1.ko 2024-08-17T02:34:32.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/raid10.ko 2024-08-17T02:34:32.241 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/raid456.ko 2024-08-17T02:34:32.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message 2024-08-17T02:34:32.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion 2024-08-17T02:34:32.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptbase.ko 2024-08-17T02:34:32.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptctl.ko 2024-08-17T02:34:32.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptfc.ko 2024-08-17T02:34:32.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptlan.ko 2024-08-17T02:34:32.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptsas.ko 2024-08-17T02:34:32.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptscsih.ko 2024-08-17T02:34:32.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptspi.ko 2024-08-17T02:34:32.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd 2024-08-17T02:34:32.242 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/lpc_ich.ko 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/lpc_sch.ko 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/mfd-core.ko 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/sm501.ko 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/viperboard.ko 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/vx855.ko 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/wl1273-core.ko 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/altera-stapl 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/apds9802als.ko 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/apds990x.ko 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/bh1770glc.ko 2024-08-17T02:34:32.243 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/cb710 2024-08-17T02:34:32.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/cb710/cb710.ko 2024-08-17T02:34:32.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/eeprom 2024-08-17T02:34:32.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/eeprom/at24.ko 2024-08-17T02:34:32.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-08-17T02:34:32.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/eeprom/max6875.ko 2024-08-17T02:34:32.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/enclosure.ko 2024-08-17T02:34:32.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/hpilo.ko 2024-08-17T02:34:32.244 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/isl29003.ko 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/isl29020.ko 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/lis3lv02d 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/mei 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/mei/mei-me.ko 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/mei/mei-txe.ko 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/mei/mei.ko 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-gru 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-gru/gru.ko 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-xp 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-xp/xp.ko 2024-08-17T02:34:32.245 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-xp/xpc.ko 2024-08-17T02:34:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-08-17T02:34:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/tifm_7xx1.ko 2024-08-17T02:34:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/tifm_core.ko 2024-08-17T02:34:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/tsl2550.ko 2024-08-17T02:34:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/vmw_balloon.ko 2024-08-17T02:34:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/vmw_vmci 2024-08-17T02:34:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-08-17T02:34:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net 2024-08-17T02:34:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/bonding 2024-08-17T02:34:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/bonding/bonding.ko 2024-08-17T02:34:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa 2024-08-17T02:34:32.246 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/b53 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/b53/b53_common.ko 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/bcm-sf2.ko 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/mv88e6060.ko 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/mv88e6xxx 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dummy.ko 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/eql.ko 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390 2024-08-17T02:34:32.247 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390/8390.ko 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/adaptec 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/agere 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/agere/et131x.ko 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/alteon 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/altera 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/amd 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-08-17T02:34:32.248 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/alx 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atl1c 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atl1e 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atlx 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom 2024-08-17T02:34:32.249 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-08-17T02:34:32.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-08-17T02:34:32.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-08-17T02:34:32.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-08-17T02:34:32.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-08-17T02:34:32.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/genet 2024-08-17T02:34:32.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-08-17T02:34:32.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-08-17T02:34:32.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/brocade 2024-08-17T02:34:32.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/brocade/bna 2024-08-17T02:34:32.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-08-17T02:34:32.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio 2024-08-17T02:34:32.250 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/cisco 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/cisco/enic 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec 2024-08-17T02:34:32.251 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dlink 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dnet.ko 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/emulex 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/emulex/benet 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/ethoc.ko 2024-08-17T02:34:32.252 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/fealnx.ko 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e100.ko 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e1000 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e1000e 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/fm10k 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/i40e 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/iavf 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/igb 2024-08-17T02:34:32.253 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-08-17T02:34:32.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/igbvf 2024-08-17T02:34:32.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-08-17T02:34:32.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/ixgbe 2024-08-17T02:34:32.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-08-17T02:34:32.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/ixgbevf 2024-08-17T02:34:32.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-08-17T02:34:32.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/libeth 2024-08-17T02:34:32.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-08-17T02:34:32.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/libie 2024-08-17T02:34:32.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-08-17T02:34:32.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/jme.ko 2024-08-17T02:34:32.254 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/marvell 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/marvell/skge.ko 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx4 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx5 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/micrel 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-08-17T02:34:32.255 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/myricom 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/myricom/myri10ge 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/natsemi 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/neterion 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/nvidia 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/packetengines 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic 2024-08-17T02:34:32.256 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/netxen 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/rdc 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek/atp.ko 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/rocker 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-08-17T02:34:32.257 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sfc 2024-08-17T02:34:32.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-08-17T02:34:32.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/silan 2024-08-17T02:34:32.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-08-17T02:34:32.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sis 2024-08-17T02:34:32.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sis/sis190.ko 2024-08-17T02:34:32.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sis/sis900.ko 2024-08-17T02:34:32.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc 2024-08-17T02:34:32.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-08-17T02:34:32.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-08-17T02:34:32.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-08-17T02:34:32.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-08-17T02:34:32.258 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/stmicro 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/stmicro/stmmac 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun/cassini.ko 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun/niu.ko 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun/sungem.ko 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/tehuti 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/ti 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/ti/tlan.ko 2024-08-17T02:34:32.259 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/via 2024-08-17T02:34:32.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-08-17T02:34:32.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-08-17T02:34:32.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/wiznet 2024-08-17T02:34:32.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-08-17T02:34:32.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-08-17T02:34:32.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/xircom 2024-08-17T02:34:32.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-08-17T02:34:32.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/geneve.ko 2024-08-17T02:34:32.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/hyperv 2024-08-17T02:34:32.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-08-17T02:34:32.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ieee802154 2024-08-17T02:34:32.260 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ieee802154/fakelb.ko 2024-08-17T02:34:32.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ifb.ko 2024-08-17T02:34:32.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ipvlan 2024-08-17T02:34:32.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ipvlan/ipvlan.ko 2024-08-17T02:34:32.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/macvlan.ko 2024-08-17T02:34:32.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/macvtap.ko 2024-08-17T02:34:32.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio 2024-08-17T02:34:32.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio.ko 2024-08-17T02:34:32.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-08-17T02:34:32.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-08-17T02:34:32.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio/mdio-regmap.ko 2024-08-17T02:34:32.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mii.ko 2024-08-17T02:34:32.261 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/net_failover.ko 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/netconsole.ko 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/nlmon.ko 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/pcs 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/pcs/pcs-lynx.ko 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/amd.ko 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/bcm7xxx.ko 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/bcm87xx.ko 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/broadcom.ko 2024-08-17T02:34:32.262 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/cicada.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/davicom.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/dp83640.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/et1011c.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/icplus.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/lxt.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/marvell.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/micrel.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/national.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/phylink.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/qsemi.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/realtek.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/smsc.ko 2024-08-17T02:34:32.263 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/ste10Xp.ko 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/vitesse.ko 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/bsd_comp.ko 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_async.ko 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_deflate.ko 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_generic.ko 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_mppe.ko 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_synctty.ko 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/pppoe.ko 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/pppox.ko 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/pptp.ko 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/slip 2024-08-17T02:34:32.264 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/slip/slhc.ko 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/slip/slip.ko 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/sungem_phy.ko 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/tap.ko 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team.ko 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_activebackup.ko 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_broadcast.ko 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_random.ko 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/tun.ko 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/veth.ko 2024-08-17T02:34:32.265 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/virtio_net.ko 2024-08-17T02:34:32.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/vmxnet3 2024-08-17T02:34:32.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-08-17T02:34:32.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/vxlan 2024-08-17T02:34:32.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/vxlan/vxlan.ko 2024-08-17T02:34:32.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/xen-netback 2024-08-17T02:34:32.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/xen-netback/xen-netback.ko 2024-08-17T02:34:32.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/xen-netfront.ko 2024-08-17T02:34:32.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ntb 2024-08-17T02:34:32.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ntb/ntb.ko 2024-08-17T02:34:32.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/nvme 2024-08-17T02:34:32.266 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/nvme/host 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/nvme/host/nvme-core.ko 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/nvme/host/nvme.ko 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport/parport.ko 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport/parport_cs.ko 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport/parport_pc.ko 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport/parport_serial.ko 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/hotplug 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/pcie 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/pcie/aer_inject.ko 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/xen-pcifront.ko 2024-08-17T02:34:32.267 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pcmcia 2024-08-17T02:34:32.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pcmcia/i82092.ko 2024-08-17T02:34:32.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pcmcia/pd6729.ko 2024-08-17T02:34:32.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pcmcia/yenta_socket.ko 2024-08-17T02:34:32.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pinctrl 2024-08-17T02:34:32.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pinctrl/intel 2024-08-17T02:34:32.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-08-17T02:34:32.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform 2024-08-17T02:34:32.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86 2024-08-17T02:34:32.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/acer-wmi.ko 2024-08-17T02:34:32.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/acerhdf.ko 2024-08-17T02:34:32.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/apple-gmux.ko 2024-08-17T02:34:32.268 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/asus-laptop.ko 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/asus-wmi.ko 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/classmate-laptop.ko 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/hdaps.ko 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/intel 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/intel_ips.ko 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/msi-wmi.ko 2024-08-17T02:34:32.269 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/mxm-wmi.ko 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/samsung-laptop.ko 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/samsung-q10.ko 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/topstar-laptop.ko 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/toshiba_haps.ko 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/wmi-bmof.ko 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/wmi.ko 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/powercap 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/powercap/intel_rapl_common.ko 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/powercap/intel_rapl_msr.ko 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/clients 2024-08-17T02:34:32.270 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/clients/pps-gpio.ko 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/clients/pps-ldisc.ko 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/clients/pps_parport.ko 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/pps_core.ko 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ptp 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ptp/ptp.ko 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ptp/ptp_kvm.ko 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras/amd 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras/amd/atl 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras/amd/fmpm.ko 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc 2024-08-17T02:34:32.271 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-bq32k.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1286.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1307.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1374.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1511.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1553.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1672.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1742.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds2404.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds3232.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-em3027.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-fm3130.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-isl12022.ko 2024-08-17T02:34:32.272 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-isl1208.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-m41t80.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-m48t35.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-m48t59.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-max6900.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-msm6242.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf2127.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf85063.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf8523.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf8563.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf8583.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rp5c01.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rs5c372.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rx8025.ko 2024-08-17T02:34:32.273 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rx8581.ko 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-x1205.ko 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/3w-9xxx.ko 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/3w-sas.ko 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/3w-xxxx.ko 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/BusLogic.ko 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/a100u2w.ko 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aacraid 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aacraid/aacraid.ko 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/advansys.ko 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aic7xxx 2024-08-17T02:34:32.274 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-08-17T02:34:32.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-08-17T02:34:32.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/am53c974.ko 2024-08-17T02:34:32.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/arcmsr 2024-08-17T02:34:32.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-08-17T02:34:32.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/atp870u.ko 2024-08-17T02:34:32.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/be2iscsi 2024-08-17T02:34:32.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-08-17T02:34:32.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bfa 2024-08-17T02:34:32.275 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bfa/bfa.ko 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bnx2fc 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bnx2i 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/ch.ko 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/csiostor 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/csiostor/csiostor.ko 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/cxgb3i 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/cxgb4i 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/dc395x.ko 2024-08-17T02:34:32.276 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler 2024-08-17T02:34:32.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-08-17T02:34:32.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-08-17T02:34:32.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-08-17T02:34:32.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-08-17T02:34:32.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/dmx3191d.ko 2024-08-17T02:34:32.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/esas2r 2024-08-17T02:34:32.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/esas2r/esas2r.ko 2024-08-17T02:34:32.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/esp_scsi.ko 2024-08-17T02:34:32.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fcoe 2024-08-17T02:34:32.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fcoe/fcoe.ko 2024-08-17T02:34:32.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-08-17T02:34:32.277 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fnic 2024-08-17T02:34:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fnic/fnic.ko 2024-08-17T02:34:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/hpsa.ko 2024-08-17T02:34:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/hptiop.ko 2024-08-17T02:34:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/hv_storvsc.ko 2024-08-17T02:34:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/initio.ko 2024-08-17T02:34:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/ipr.ko 2024-08-17T02:34:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/ips.ko 2024-08-17T02:34:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/isci 2024-08-17T02:34:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/isci/isci.ko 2024-08-17T02:34:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-08-17T02:34:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/iscsi_tcp.ko 2024-08-17T02:34:32.278 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libfc 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libfc/libfc.ko 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libiscsi.ko 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libiscsi_tcp.ko 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libsas 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libsas/libsas.ko 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/lpfc 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/lpfc/lpfc.ko 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid.ko 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-08-17T02:34:32.279 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mpt3sas 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mvsas 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mvsas/mvsas.ko 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mvumi.ko 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/pm8001 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/pmcraid.ko 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla1280.ko 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla2xxx 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla4xxx 2024-08-17T02:34:32.280 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/raid_class.ko 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_debug.ko 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_fc.ko 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_sas.ko 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_spi.ko 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_srp.ko 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/ses.ko 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/st.ko 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/stex.ko 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/sym53c8xx_2 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-08-17T02:34:32.281 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/virtio_scsi.ko 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/vmw_pvscsi.ko 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/wd719x.ko 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ssb 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ssb/ssb.ko 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/iscsi 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/loopback 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/loopback/tcm_loop.ko 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_file.ko 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_iblock.ko 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_mod.ko 2024-08-17T02:34:32.282 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_pscsi.ko 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_user.ko 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/tcm_fc 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-08-17T02:34:32.283 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/ipwireless 2024-08-17T02:34:32.284 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/n_gsm.ko 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/n_hdlc.ko 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/nozomi.ko 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/8250 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/arc_uart.ko 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/jsm 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/jsm/jsm.ko 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs/core 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs/core/ufshcd-core.ko 2024-08-17T02:34:32.285 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs/host 2024-08-17T02:34:32.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-08-17T02:34:32.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio 2024-08-17T02:34:32.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio.ko 2024-08-17T02:34:32.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio_aec.ko 2024-08-17T02:34:32.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio_cif.ko 2024-08-17T02:34:32.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio_pci_generic.ko 2024-08-17T02:34:32.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio_sercos3.ko 2024-08-17T02:34:32.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb 2024-08-17T02:34:32.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class 2024-08-17T02:34:32.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class/cdc-acm.ko 2024-08-17T02:34:32.286 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class/cdc-wdm.ko 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class/usblp.ko 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class/usbtmc.ko 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/host 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/host/sl811-hcd.ko 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/image 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/image/mdc800.ko 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/image/microtek.ko 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/adutux.ko 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/appledisplay.ko 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/emi26.ko 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/emi62.ko 2024-08-17T02:34:32.287 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/ezusb.ko 2024-08-17T02:34:32.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/idmouse.ko 2024-08-17T02:34:32.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/iowarrior.ko 2024-08-17T02:34:32.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/isight_firmware.ko 2024-08-17T02:34:32.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/ldusb.ko 2024-08-17T02:34:32.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/legousbtower.ko 2024-08-17T02:34:32.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/sisusbvga 2024-08-17T02:34:32.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-08-17T02:34:32.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/trancevibrator.ko 2024-08-17T02:34:32.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/usb3503.ko 2024-08-17T02:34:32.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/usblcd.ko 2024-08-17T02:34:32.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/usbsevseg.ko 2024-08-17T02:34:32.288 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/uss720.ko 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/yurex.ko 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/phy 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/phy/phy-generic.ko 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/aircable.ko 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ark3116.ko 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/belkin_sa.ko 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ch341.ko 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/cp210x.ko 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/cyberjack.ko 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/cypress_m8.ko 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/digi_acceleport.ko 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/empeg.ko 2024-08-17T02:34:32.289 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ftdi_sio.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/garmin_gps.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/io_edgeport.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/io_ti.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ipaq.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ipw.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ir-usb.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/keyspan.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/keyspan_pda.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/kl5kusb105.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/kobil_sct.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/mct_u232.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/mos7720.ko 2024-08-17T02:34:32.290 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/mos7840.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/navman.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/omninet.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/opticon.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/option.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/oti6858.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/pl2303.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/qcaux.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/qcserial.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/quatech2.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/safe_serial.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/sierra.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/spcp8x5.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ssu100.ko 2024-08-17T02:34:32.291 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/symbolserial.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/usb_debug.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/usb_wwan.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/visor.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/whiteheat.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/xsens_mt.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/uas.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-alauda.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-cypress.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-datafab.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-08-17T02:34:32.292 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-freecom.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-isd200.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-karma.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-onetouch.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-realtek.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-sddr09.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-sddr55.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-usbat.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/usb-storage.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/usbip 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/usbip/usbip-core.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/usbip/usbip-host.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio 2024-08-17T02:34:32.293 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/pci 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/pci/vfio-pci.ko 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/vfio.ko 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost/vhost.ko 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost/vhost_iotlb.ko 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost/vhost_net.ko 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost/vhost_scsi.ko 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video/backlight 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video/backlight/apple_bl.ko 2024-08-17T02:34:32.294 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video/backlight/lcd.ko 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video/backlight/platform_lcd.ko 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio.ko 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_balloon.ko 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_mmio.ko 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_pci.ko 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_ring.ko 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/masters 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/masters/ds2482.ko 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/masters/ds2490.ko 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves 2024-08-17T02:34:32.295 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_smem.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_therm.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/wire.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/alim1535_wdt.ko 2024-08-17T02:34:32.296 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/alim7101_wdt.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/f71808e_wdt.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/hpwdt.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/i6300esb.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/iTCO_wdt.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/ib700wdt.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/ibmasr.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/it8712f_wdt.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/it87_wdt.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/machzwd.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/nv_tco.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/pcwd_pci.ko 2024-08-17T02:34:32.297 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/pcwd_usb.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/sch311x_wdt.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/softdog.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/sp5100_tco.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/via_wdt.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/w83877f_wdt.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/w83977f_wdt.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/wdt_pci.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/xen_wdt.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-acpi-processor.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-evtchn.ko 2024-08-17T02:34:32.298 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-gntalloc.ko 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-gntdev.ko 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-pciback 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-privcmd.ko 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xenfs 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xenfs/xenfs.ko 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/9p 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/9p/9p.ko 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/affs 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/affs/affs.ko 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/befs 2024-08-17T02:34:32.299 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/befs/befs.ko 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/binfmt_misc.ko 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/btrfs 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/btrfs/btrfs.ko 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/cachefiles 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/cachefiles/cachefiles.ko 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ceph 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ceph/ceph.ko 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/coda 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/coda/coda.ko 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/cramfs 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/cramfs/cramfs.ko 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/dlm 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/dlm/dlm.ko 2024-08-17T02:34:32.300 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ecryptfs 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ecryptfs/ecryptfs.ko 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/f2fs 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/f2fs/f2fs.ko 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fat 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fat/fat.ko 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fat/msdos.ko 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fat/vfat.ko 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fuse 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fuse/cuse.ko 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fuse/fuse.ko 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/gfs2 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/gfs2/gfs2.ko 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/hfs 2024-08-17T02:34:32.301 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/hfs/hfs.ko 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/hfsplus 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/hfsplus/hfsplus.ko 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/isofs 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/isofs/isofs.ko 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/jfs 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/jfs/jfs.ko 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/lockd 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/lockd/lockd.ko 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/minix 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/minix/minix.ko 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/netfs 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/netfs/netfs.ko 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs 2024-08-17T02:34:32.302 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/blocklayout 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/filelayout 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/flexfilelayout 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/nfs.ko 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/nfsv3.ko 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/nfsv4.ko 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs_common 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs_common/grace.ko 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs_common/nfs_acl.ko 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfsd 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfsd/nfsd.ko 2024-08-17T02:34:32.303 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nilfs2 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nilfs2/nilfs2.ko 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-celtic.ko 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-centeuro.ko 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-croatian.ko 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-cyrillic.ko 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-gaelic.ko 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-greek.ko 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-iceland.ko 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-inuit.ko 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-roman.ko 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-romanian.ko 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-turkish.ko 2024-08-17T02:34:32.304 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp1250.ko 2024-08-17T02:34:32.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp1251.ko 2024-08-17T02:34:32.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp1255.ko 2024-08-17T02:34:32.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp737.ko 2024-08-17T02:34:32.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp775.ko 2024-08-17T02:34:32.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp850.ko 2024-08-17T02:34:32.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp852.ko 2024-08-17T02:34:32.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp855.ko 2024-08-17T02:34:32.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp857.ko 2024-08-17T02:34:32.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp860.ko 2024-08-17T02:34:32.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp861.ko 2024-08-17T02:34:32.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp862.ko 2024-08-17T02:34:32.305 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp863.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp864.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp865.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp866.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp869.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp874.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp932.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp936.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp949.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp950.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_euc-jp.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-1.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-13.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-14.ko 2024-08-17T02:34:32.306 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-15.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-2.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-3.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-4.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-5.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-6.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-7.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-9.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_koi8-r.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_koi8-ru.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_koi8-u.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_ucs2_utils.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_utf8.ko 2024-08-17T02:34:32.307 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/cluster 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/dlm 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/dlmfs 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/ocfs2.ko 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/overlayfs 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/overlayfs/overlay.ko 2024-08-17T02:34:32.308 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/pstore 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/pstore/ramoops.ko 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/reiserfs 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/reiserfs/reiserfs.ko 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/romfs 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/romfs/romfs.ko 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/client 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/client/cifs.ko 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/common 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/common/cifs_arc4.ko 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/common/cifs_md4.ko 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/squashfs 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/squashfs/squashfs.ko 2024-08-17T02:34:32.309 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/sysv 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/sysv/sysv.ko 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/udf 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/udf/udf.ko 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ufs 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ufs/ufs.ko 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/xfs 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/xfs/xfs.ko 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/rcu 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/rcu/rcutorture.ko 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/torture.ko 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/trace 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib 2024-08-17T02:34:32.310 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/asn1_encoder.ko 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crc-itu-t.ko 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crc8.ko 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crypto 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crypto/libaescfb.ko 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crypto/libarc4.ko 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crypto/libdes.ko 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/libcrc32c.ko 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/lru_cache.ko 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/lz4 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/lz4/lz4_compress.ko 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/lz4/lz4hc_compress.ko 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/math 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/math/cordic.ko 2024-08-17T02:34:32.311 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/raid6 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/raid6/raid6_pq.ko 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/reed_solomon 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/reed_solomon/reed_solomon.ko 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/ts_bm.ko 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/ts_fsm.ko 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/ts_kmp.ko 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/zstd 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/zstd/zstd_compress.ko 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/mm 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/mm/hwpoison-inject.ko 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/6lowpan.ko 2024-08-17T02:34:32.312 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_dest.ko 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_fragment.ko 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_hop.ko 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_ipv6.ko 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_mobility.ko 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_routing.ko 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_udp.ko 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/garp.ko 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/mrp.ko 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/p8022.ko 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/psnap.ko 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/stp.ko 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/8021q 2024-08-17T02:34:32.313 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/8021q/8021q.ko 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p/9pnet.ko 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p/9pnet_fd.ko 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p/9pnet_rdma.ko 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p/9pnet_virtio.ko 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/batman-adv 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/batman-adv/batman-adv.ko 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/br_netfilter.ko 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/bridge.ko 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-08-17T02:34:32.314 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_among.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_arp.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_ip.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_limit.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_log.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_mark.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-08-17T02:34:32.315 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_snat.ko 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_stp.ko 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebtables.ko 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ceph 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ceph/libceph.ko 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/core 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/core/failover.ko 2024-08-17T02:34:32.316 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/core/pktgen.ko 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp/dccp.ko 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp/dccp_diag.ko 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp/dccp_ipv4.ko 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp/dccp_ipv6.ko 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dns_resolver 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dns_resolver/dns_resolver.ko 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/dsa_core.ko 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/tag_brcm.ko 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/tag_dsa.ko 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/tag_none.ko 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/tag_trailer.ko 2024-08-17T02:34:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154 2024-08-17T02:34:32.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154/6lowpan 2024-08-17T02:34:32.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-08-17T02:34:32.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154/ieee802154.ko 2024-08-17T02:34:32.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154/ieee802154_socket.ko 2024-08-17T02:34:32.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4 2024-08-17T02:34:32.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ah4.ko 2024-08-17T02:34:32.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/esp4.ko 2024-08-17T02:34:32.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/fou.ko 2024-08-17T02:34:32.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/gre.ko 2024-08-17T02:34:32.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/inet_diag.ko 2024-08-17T02:34:32.318 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ip_gre.ko 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ip_tunnel.ko 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ip_vti.ko 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ipcomp.ko 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ipip.ko 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/arp_tables.ko 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-08-17T02:34:32.319 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/iptable_security.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-08-17T02:34:32.320 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_bic.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_dctcp.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_diag.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_highspeed.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_htcp.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_hybla.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_illinois.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_lp.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_scalable.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_vegas.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_veno.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_westwood.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_yeah.ko 2024-08-17T02:34:32.321 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tunnel4.ko 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/udp_diag.ko 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/udp_tunnel.ko 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/xfrm4_tunnel.ko 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ah6.ko 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/esp6.ko 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/fou6.ko 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ip6_tunnel.ko 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ip6_vti.ko 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ipcomp6.ko 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter 2024-08-17T02:34:32.322 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-08-17T02:34:32.323 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/sit.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/tunnel6.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/xfrm6_tunnel.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/key 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/key/af_key.ko 2024-08-17T02:34:32.324 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_core.ko 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_debugfs.ko 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_eth.ko 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_ip.ko 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_ip6.ko 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_netlink.ko 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_ppp.ko 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/llc 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/llc/llc.ko 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/mac802154 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/mac802154/mac802154.ko 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/mpls 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/mpls/mpls_gso.ko 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter 2024-08-17T02:34:32.325 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set.ko 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-08-17T02:34:32.326 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs.ko 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-08-17T02:34:32.327 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conncount.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_h323.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_irc.ko 2024-08-17T02:34:32.328 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-08-17T02:34:32.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-08-17T02:34:32.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-08-17T02:34:32.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_sane.ko 2024-08-17T02:34:32.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_sip.ko 2024-08-17T02:34:32.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-08-17T02:34:32.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-08-17T02:34:32.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_dup_netdev.ko 2024-08-17T02:34:32.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_flow_table.ko 2024-08-17T02:34:32.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_flow_table_inet.ko 2024-08-17T02:34:32.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_log_syslog.ko 2024-08-17T02:34:32.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat.ko 2024-08-17T02:34:32.329 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_amanda.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_ftp.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_irc.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_sip.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_tftp.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_synproxy_core.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_tables.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink_acct.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink_log.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink_osf.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink_queue.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_chain_nat.ko 2024-08-17T02:34:32.330 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_compat.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_ct.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_dup_netdev.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_fib.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_fwd_netdev.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_hash.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_limit.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_log.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_masq.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_nat.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_numgen.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_queue.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_quota.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_redir.ko 2024-08-17T02:34:32.331 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_reject.ko 2024-08-17T02:34:32.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_reject_inet.ko 2024-08-17T02:34:32.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_socket.ko 2024-08-17T02:34:32.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_tunnel.ko 2024-08-17T02:34:32.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_AUDIT.ko 2024-08-17T02:34:32.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_CHECKSUM.ko 2024-08-17T02:34:32.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_CLASSIFY.ko 2024-08-17T02:34:32.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-08-17T02:34:32.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_CT.ko 2024-08-17T02:34:32.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_DSCP.ko 2024-08-17T02:34:32.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_HL.ko 2024-08-17T02:34:32.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_HMARK.ko 2024-08-17T02:34:32.332 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_IDLETIMER.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_LED.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_LOG.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_MASQUERADE.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_NETMAP.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_NFLOG.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_NFQUEUE.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_RATEEST.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_REDIRECT.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_SECMARK.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TCPMSS.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TEE.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TPROXY.ko 2024-08-17T02:34:32.333 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TRACE.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_addrtype.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_bpf.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_cgroup.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_cluster.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_comment.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_connbytes.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_connlabel.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_connlimit.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_connmark.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_conntrack.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_cpu.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_dccp.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_devgroup.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_dscp.ko 2024-08-17T02:34:32.334 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_ecn.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_esp.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_hashlimit.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_helper.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_hl.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_ipcomp.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_iprange.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_ipvs.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_l2tp.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_length.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_limit.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_mac.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_mark.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_multiport.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_nat.ko 2024-08-17T02:34:32.335 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_nfacct.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_osf.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_owner.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_physdev.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_pkttype.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_policy.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_quota.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_rateest.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_realm.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_recent.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_sctp.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_set.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_socket.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_state.ko 2024-08-17T02:34:32.336 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_statistic.ko 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_string.ko 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_tcpmss.ko 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_time.ko 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_u32.ko 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netlink 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netlink/netlink_diag.ko 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/nsh 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/nsh/nsh.ko 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch/openvswitch.ko 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch/vport-geneve.ko 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch/vport-gre.ko 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch/vport-vxlan.ko 2024-08-17T02:34:32.337 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/packet 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/packet/af_packet_diag.ko 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/rds 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/rds/rds.ko 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/rds/rds_rdma.ko 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/rds/rds_tcp.ko 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_csum.ko 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_gact.ko 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_mirred.ko 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_nat.ko 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_pedit.ko 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_police.ko 2024-08-17T02:34:32.338 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_simple.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_skbedit.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_vlan.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_basic.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_bpf.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_flow.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_fw.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_route.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_u32.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_cmp.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_ipset.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_meta.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_nbyte.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_text.ko 2024-08-17T02:34:32.339 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_u32.ko 2024-08-17T02:34:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_choke.ko 2024-08-17T02:34:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_codel.ko 2024-08-17T02:34:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_drr.ko 2024-08-17T02:34:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_fq.ko 2024-08-17T02:34:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_gred.ko 2024-08-17T02:34:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_hfsc.ko 2024-08-17T02:34:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_hhf.ko 2024-08-17T02:34:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_htb.ko 2024-08-17T02:34:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_ingress.ko 2024-08-17T02:34:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_mqprio.ko 2024-08-17T02:34:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_mqprio_lib.ko 2024-08-17T02:34:32.340 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_multiq.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_netem.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_pie.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_plug.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_prio.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_qfq.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_red.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_sfb.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_sfq.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_tbf.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_teql.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sctp 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sctp/sctp.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sctp/sctp_diag.ko 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc 2024-08-17T02:34:32.341 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/auth_gss 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/sunrpc.ko 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/xprtrdma 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/tipc 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/tipc/tipc.ko 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/tipc/tipc_diag.ko 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/unix 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/unix/unix_diag.ko 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vsock.ko 2024-08-17T02:34:32.342 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vsock_diag.ko 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vsock_loopback.ko 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/xfrm 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/xfrm/xfrm_ipcomp.ko 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys/encrypted-keys 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys/trusted-keys 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys/trusted-keys/trusted.ko 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.alias 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.alias.bin 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.builtin 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.builtin.alias.bin 2024-08-17T02:34:32.343 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.builtin.bin 2024-08-17T02:34:32.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.builtin.modinfo 2024-08-17T02:34:32.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.dep 2024-08-17T02:34:32.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.dep.bin 2024-08-17T02:34:32.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.devname 2024-08-17T02:34:32.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.order 2024-08-17T02:34:32.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.softdep 2024-08-17T02:34:32.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.symbols 2024-08-17T02:34:32.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.symbols.bin 2024-08-17T02:34:32.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.weakdep 2024-08-17T02:34:32.344 INFO:teuthology.orchestra.run.smithi142.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/vmlinuz 2024-08-17T02:34:32.348 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:32.348 DEBUG:teuthology.orchestra.run.smithi142:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-08-17T02:34:32.373 INFO:teuthology.orchestra.run.smithi142.stdout:postinstall scriptlet (using /bin/sh): 2024-08-17T02:34:32.373 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-17T02:34:32.373 INFO:teuthology.orchestra.run.smithi142.stdout: /usr/bin/kernel-install add 6.11.0-rc3-g7503b862ff5b /lib/modules/6.11.0-rc3-g7503b862ff5b/vmlinuz 2024-08-17T02:34:32.373 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2024-08-17T02:34:32.373 INFO:teuthology.orchestra.run.smithi142.stdout:for file in vmlinuz System.map config; do 2024-08-17T02:34:32.373 INFO:teuthology.orchestra.run.smithi142.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc3-g7503b862ff5b/${file}" "/boot/${file}-6.11.0-rc3-g7503b862ff5b"; then 2024-08-17T02:34:32.373 INFO:teuthology.orchestra.run.smithi142.stdout: cp "/lib/modules/6.11.0-rc3-g7503b862ff5b/${file}" "/boot/${file}-6.11.0-rc3-g7503b862ff5b" 2024-08-17T02:34:32.373 INFO:teuthology.orchestra.run.smithi142.stdout: fi 2024-08-17T02:34:32.373 INFO:teuthology.orchestra.run.smithi142.stdout:done 2024-08-17T02:34:32.373 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -d "/lib/modules/6.11.0-rc3-g7503b862ff5b/dtb" ] && \ 2024-08-17T02:34:32.373 INFO:teuthology.orchestra.run.smithi142.stdout: ! diff -rq "/lib/modules/6.11.0-rc3-g7503b862ff5b/dtb" "/boot/dtb-6.11.0-rc3-g7503b862ff5b" >/dev/null 2>&1; then 2024-08-17T02:34:32.374 INFO:teuthology.orchestra.run.smithi142.stdout: rm -rf "/boot/dtb-6.11.0-rc3-g7503b862ff5b" 2024-08-17T02:34:32.374 INFO:teuthology.orchestra.run.smithi142.stdout: cp -r "/lib/modules/6.11.0-rc3-g7503b862ff5b/dtb" "/boot/dtb-6.11.0-rc3-g7503b862ff5b" 2024-08-17T02:34:32.374 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2024-08-17T02:34:32.374 INFO:teuthology.orchestra.run.smithi142.stdout:if [ ! -e "/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.dep" ]; then 2024-08-17T02:34:32.374 INFO:teuthology.orchestra.run.smithi142.stdout: /usr/sbin/depmod 6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:32.374 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2024-08-17T02:34:32.374 INFO:teuthology.orchestra.run.smithi142.stdout:preuninstall scriptlet (using /bin/sh): 2024-08-17T02:34:32.374 INFO:teuthology.orchestra.run.smithi142.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-17T02:34:32.374 INFO:teuthology.orchestra.run.smithi142.stdout:kernel-install remove 6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:32.374 INFO:teuthology.orchestra.run.smithi142.stdout:fi 2024-08-17T02:34:32.375 DEBUG:teuthology.orchestra.run.smithi142:> sudo rpm -qi grub2-tools 2024-08-17T02:34:32.456 INFO:teuthology.orchestra.run.smithi142.stdout:Name : grub2-tools 2024-08-17T02:34:32.456 INFO:teuthology.orchestra.run.smithi142.stdout:Epoch : 1 2024-08-17T02:34:32.456 INFO:teuthology.orchestra.run.smithi142.stdout:Version : 2.06 2024-08-17T02:34:32.456 INFO:teuthology.orchestra.run.smithi142.stdout:Release : 68.el9 2024-08-17T02:34:32.456 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: x86_64 2024-08-17T02:34:32.456 INFO:teuthology.orchestra.run.smithi142.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-08-17T02:34:32.456 INFO:teuthology.orchestra.run.smithi142.stdout:Group : Unspecified 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:Size : 8264254 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:License : GPLv3+ 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:Packager : builder@centos.org 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:Vendor : CentOS 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:URL : http://www.gnu.org/software/grub/ 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:Summary : Support tools for GRUB. 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:Description : 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-08-17T02:34:32.457 INFO:teuthology.orchestra.run.smithi142.stdout:customizable bootloader with modular architecture. It supports a rich 2024-08-17T02:34:32.458 INFO:teuthology.orchestra.run.smithi142.stdout:variety of kernel formats, file systems, computer architectures and 2024-08-17T02:34:32.458 INFO:teuthology.orchestra.run.smithi142.stdout:hardware devices. 2024-08-17T02:34:32.458 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:34:32.458 INFO:teuthology.orchestra.run.smithi142.stdout:This subpackage provides tools for support of all platforms. 2024-08-17T02:34:32.459 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-08-17T02:34:32.459 INFO:teuthology.task.kernel:Updating grub on smithi142 to boot 6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:32.459 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-08-17T02:34:33.020 INFO:teuthology.orchestra.run.smithi142.stderr:Generating grub configuration file ... 2024-08-17T02:34:34.447 INFO:teuthology.orchestra.run.smithi142.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-08-17T02:34:34.473 INFO:teuthology.orchestra.run.smithi142.stderr:done 2024-08-17T02:34:34.475 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-08-17T02:34:34.494 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.rVBElmc6AC 2024-08-17T02:34:34.495 DEBUG:teuthology.orchestra.run.smithi142:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rVBElmc6AC 2024-08-17T02:34:34.565 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.rVBElmc6AC 2024-08-17T02:34:34.696 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.rVBElmc6AC is 6KB 2024-08-17T02:34:34.745 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.rVBElmc6AC 2024-08-17T02:34:34.759 DEBUG:teuthology.orchestra.run.smithi142:> sudo /bin/ls /boot/loader/entries || true 2024-08-17T02:34:34.820 INFO:teuthology.orchestra.run.smithi142.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-08-17T02:34:34.820 INFO:teuthology.orchestra.run.smithi142.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-08-17T02:34:34.820 INFO:teuthology.orchestra.run.smithi142.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-08-17T02:34:34.820 INFO:teuthology.orchestra.run.smithi142.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.11.0-rc3-g7503b862ff5b.conf 2024-08-17T02:34:34.821 DEBUG:teuthology.orchestra.run.smithi142:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:34.959 DEBUG:teuthology.orchestra.run.smithi142:> sudo shutdown -r now 2024-08-17T02:34:39.188 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-08-17T02:34:39.188 DEBUG:teuthology.orchestra.run.smithi016:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-08-17T02:34:39.266 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/System.map-6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:39.266 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/config-6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:39.266 INFO:teuthology.orchestra.run.smithi016.stdout:/boot/vmlinuz-6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:39.266 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/System.map 2024-08-17T02:34:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/config 2024-08-17T02:34:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel 2024-08-17T02:34:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch 2024-08-17T02:34:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86 2024-08-17T02:34:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto 2024-08-17T02:34:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-08-17T02:34:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-08-17T02:34:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-08-17T02:34:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-08-17T02:34:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-08-17T02:34:39.267 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-08-17T02:34:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-08-17T02:34:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/crc32c-intel.ko 2024-08-17T02:34:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-08-17T02:34:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-08-17T02:34:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-08-17T02:34:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-08-17T02:34:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/serpent-avx2.ko 2024-08-17T02:34:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-08-17T02:34:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-08-17T02:34:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-08-17T02:34:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-08-17T02:34:39.268 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kernel 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kernel/cpu 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kernel/cpu/mce 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kvm 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kvm/kvm-amd.ko 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kvm/kvm-intel.ko 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/arch/x86/kvm/kvm.ko 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/ansi_cprng.ko 2024-08-17T02:34:39.269 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/anubis.ko 2024-08-17T02:34:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/arc4.ko 2024-08-17T02:34:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx 2024-08-17T02:34:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_memcpy.ko 2024-08-17T02:34:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_pq.ko 2024-08-17T02:34:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_raid6_recov.ko 2024-08-17T02:34:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_tx.ko 2024-08-17T02:34:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/async_xor.ko 2024-08-17T02:34:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/async_tx/raid6test.ko 2024-08-17T02:34:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/authenc.ko 2024-08-17T02:34:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/authencesn.ko 2024-08-17T02:34:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/blake2b_generic.ko 2024-08-17T02:34:39.270 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/blowfish_common.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/blowfish_generic.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/camellia_generic.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cast5_generic.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cast6_generic.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cast_common.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/ccm.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cmac.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/crc32_generic.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/crypto_engine.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/crypto_user.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/cts.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/des_generic.ko 2024-08-17T02:34:39.271 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/dh_generic.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/ecc.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/ecdh_generic.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/echainiv.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/essiv.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/fcrypt.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/khazad.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/lz4.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/lz4hc.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/md4.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/michael_mic.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/pcbc.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/pcrypt.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/rmd160.ko 2024-08-17T02:34:39.272 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/seed.ko 2024-08-17T02:34:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/serpent_generic.ko 2024-08-17T02:34:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/tcrypt.ko 2024-08-17T02:34:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/tea.ko 2024-08-17T02:34:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/twofish_common.ko 2024-08-17T02:34:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/twofish_generic.ko 2024-08-17T02:34:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/vmac.ko 2024-08-17T02:34:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/wp512.ko 2024-08-17T02:34:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/xcbc.ko 2024-08-17T02:34:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/xor.ko 2024-08-17T02:34:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/crypto/xxhash_generic.ko 2024-08-17T02:34:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers 2024-08-17T02:34:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/acpi_ipmi.ko 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/acpi_pad.ko 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/ec_sys.ko 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/platform_profile.ko 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/sbs.ko 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/sbshc.ko 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/acpi/video.ko 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/acard-ahci.ko 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/ahci_platform.ko 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/ata_generic.ko 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/libahci_platform.ko 2024-08-17T02:34:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_acpi.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_ali.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_amd.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_artop.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_atiixp.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_atp867x.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_cmd640.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_cmd64x.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_cypress.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_efar.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_hpt366.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_hpt37x.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_hpt3x2n.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_hpt3x3.ko 2024-08-17T02:34:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_it8213.ko 2024-08-17T02:34:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_it821x.ko 2024-08-17T02:34:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_jmicron.ko 2024-08-17T02:34:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_marvell.ko 2024-08-17T02:34:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_mpiix.ko 2024-08-17T02:34:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_netcell.ko 2024-08-17T02:34:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_ninja32.ko 2024-08-17T02:34:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_ns87410.ko 2024-08-17T02:34:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_ns87415.ko 2024-08-17T02:34:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_oldpiix.ko 2024-08-17T02:34:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_opti.ko 2024-08-17T02:34:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_optidma.ko 2024-08-17T02:34:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_pcmcia.ko 2024-08-17T02:34:39.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_pdc2027x.ko 2024-08-17T02:34:39.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-08-17T02:34:39.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_piccolo.ko 2024-08-17T02:34:39.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_rdc.ko 2024-08-17T02:34:39.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_sch.ko 2024-08-17T02:34:39.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_serverworks.ko 2024-08-17T02:34:39.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_sil680.ko 2024-08-17T02:34:39.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_sis.ko 2024-08-17T02:34:39.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_sl82c105.ko 2024-08-17T02:34:39.277 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_triflex.ko 2024-08-17T02:34:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pata_via.ko 2024-08-17T02:34:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/pdc_adma.ko 2024-08-17T02:34:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_inic162x.ko 2024-08-17T02:34:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_mv.ko 2024-08-17T02:34:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_nv.ko 2024-08-17T02:34:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_promise.ko 2024-08-17T02:34:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_qstor.ko 2024-08-17T02:34:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_sil.ko 2024-08-17T02:34:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_sil24.ko 2024-08-17T02:34:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_sis.ko 2024-08-17T02:34:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_svw.ko 2024-08-17T02:34:39.278 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_sx4.ko 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_uli.ko 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_via.ko 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ata/sata_vsc.ko 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/auxdisplay 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/auxdisplay/ks0108.ko 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/base 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/base/regmap 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/base/regmap/regmap-i2c.ko 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/base/regmap/regmap-mmio.ko 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/bcma 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/bcma/bcma.ko 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block 2024-08-17T02:34:39.279 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/aoe 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/aoe/aoe.ko 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/brd.ko 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/drbd 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/drbd/drbd.ko 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/floppy.ko 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/loop.ko 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/mtip32xx 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/nbd.ko 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/null_blk 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/null_blk/null_blk.ko 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/pktcdvd.ko 2024-08-17T02:34:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/rbd.ko 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/virtio_blk.ko 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/xen-blkback 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/xen-blkfront.ko 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/zram 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/block/zram/zram.ko 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hangcheck-timer.ko 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/amd-rng.ko 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/intel-rng.ko 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-08-17T02:34:39.281 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/via-rng.ko 2024-08-17T02:34:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/hw_random/virtio-rng.ko 2024-08-17T02:34:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi 2024-08-17T02:34:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-08-17T02:34:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-08-17T02:34:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-08-17T02:34:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_si.ko 2024-08-17T02:34:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-08-17T02:34:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-08-17T02:34:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/lp.ko 2024-08-17T02:34:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/mwave 2024-08-17T02:34:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/mwave/mwave.ko 2024-08-17T02:34:39.282 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/ppdev.ko 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tlclk.ko 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm.ko 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_atmel.ko 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_infineon.ko 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_nsc.ko 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_tis.ko 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/uv_mmtimer.ko 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/char/virtio_console.ko 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-08-17T02:34:39.283 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/p4-clockmod.ko 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/powernow-k8.ko 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/cpufreq/speedstep-lib.ko 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/ccp 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/ccp/ccp.ko 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat/qat_common 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-08-17T02:34:39.284 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/padlock-aes.ko 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/padlock-sha.ko 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/virtio 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dca 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dca/dca.ko 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/devfreq 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma/dw 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma/dw/dw_dmac.ko 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma/ioat 2024-08-17T02:34:39.285 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/dma/ioat/ioatdma.ko 2024-08-17T02:34:39.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac 2024-08-17T02:34:39.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/amd64_edac.ko 2024-08-17T02:34:39.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/e752x_edac.ko 2024-08-17T02:34:39.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/edac_mce_amd.ko 2024-08-17T02:34:39.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i3000_edac.ko 2024-08-17T02:34:39.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i3200_edac.ko 2024-08-17T02:34:39.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i5100_edac.ko 2024-08-17T02:34:39.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i5400_edac.ko 2024-08-17T02:34:39.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i7300_edac.ko 2024-08-17T02:34:39.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i7core_edac.ko 2024-08-17T02:34:39.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/i82975x_edac.ko 2024-08-17T02:34:39.286 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/ie31200_edac.ko 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/sb_edac.ko 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/edac/x38_edac.ko 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/firmware 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/firmware/edd.ko 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/firmware/iscsi_ibft.ko 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/gpio 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/gpio/gpio-ich.ko 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/gpio/gpio-viperboard.ko 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-appleir.ko 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-aureal.ko 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-axff.ko 2024-08-17T02:34:39.287 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-dr.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-elecom.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-elo.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-emsff.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-gaff.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-gt683r.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-gyration.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-holtek-kbd.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-holtek-mouse.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-holtekff.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-hyperv.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-icade.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-keytouch.ko 2024-08-17T02:34:39.288 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-kye.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-lcpower.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-led.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-lenovo.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-logitech-dj.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-multitouch.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-ortek.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-penmount.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-petalynx.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-picolcd.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-pl.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-plantronics.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-primax.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-rmi.ko 2024-08-17T02:34:39.289 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-arvo.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-common.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-isku.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-kone.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-konepure.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-lua.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-pyra.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-ryos.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat-savu.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-roccat.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-saitek.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-samsung.ko 2024-08-17T02:34:39.290 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-sensor-hub.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-sjoy.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-sony.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-speedlink.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-steelseries.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-sunplus.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-thrustmaster.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-tivo.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-tmff.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-topseed.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-twinhan.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-uclogic.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-waltop.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-wiimote.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-xinmo.ko 2024-08-17T02:34:39.291 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-zpff.ko 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/hid-zydacron.ko 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/uhid.ko 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hid/wacom.ko 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hv 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hv/hv_balloon.ko 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hv/hv_utils.ko 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hv/hv_vmbus.ko 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/abituguru.ko 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/abituguru3.ko 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/acpi_power_meter.ko 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ad7414.ko 2024-08-17T02:34:39.292 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ad7418.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adc128d818.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm1025.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm1026.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm1029.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm1031.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adm9240.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ads7828.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7410.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7411.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7462.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7470.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7475.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/adt7x10.ko 2024-08-17T02:34:39.293 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/amc6821.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/applesmc.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/asb100.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/asc7621.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/asus_atk0110.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/atxp1.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/coretemp.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/dme1737.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ds1621.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ds620.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/emc1403.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/emc6w201.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/f71805f.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/f71882fg.ko 2024-08-17T02:34:39.294 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/f75375s.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/fam15h_power.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/fschmd.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/g760a.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/g762.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/gl518sm.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/gl520sm.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/hwmon-vid.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/i5500_temp.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/i5k_amb.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ibmaem.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ibmpex.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ina209.ko 2024-08-17T02:34:39.295 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ina2xx.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/it87.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/k10temp.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/k8temp.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lineage-pem.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm63.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm73.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm75.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm77.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm78.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm80.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm83.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm85.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm87.ko 2024-08-17T02:34:39.296 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm90.ko 2024-08-17T02:34:39.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm92.ko 2024-08-17T02:34:39.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm93.ko 2024-08-17T02:34:39.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm95234.ko 2024-08-17T02:34:39.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm95241.ko 2024-08-17T02:34:39.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/lm95245.ko 2024-08-17T02:34:39.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc2945.ko 2024-08-17T02:34:39.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4151.ko 2024-08-17T02:34:39.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4215.ko 2024-08-17T02:34:39.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4222.ko 2024-08-17T02:34:39.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4245.ko 2024-08-17T02:34:39.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4260.ko 2024-08-17T02:34:39.297 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/ltc4261.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max16065.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max1619.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max1668.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max197.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max6639.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max6650.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/max6697.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/mcp3021.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/nct6683.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/nct6775-core.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/nct6775.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/nct7802.ko 2024-08-17T02:34:39.298 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pc87360.ko 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pc87427.ko 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pcf8591.ko 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/max16064.ko 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/max34440.ko 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/max8688.ko 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-08-17T02:34:39.299 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-08-17T02:34:39.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-08-17T02:34:39.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-08-17T02:34:39.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/powr1220.ko 2024-08-17T02:34:39.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sch5627.ko 2024-08-17T02:34:39.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sch5636.ko 2024-08-17T02:34:39.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sch56xx-common.ko 2024-08-17T02:34:39.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sht15.ko 2024-08-17T02:34:39.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sht21.ko 2024-08-17T02:34:39.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/shtc1.ko 2024-08-17T02:34:39.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/sis5595.ko 2024-08-17T02:34:39.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/smsc47b397.ko 2024-08-17T02:34:39.300 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/smsc47m1.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/smsc47m192.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/thmc50.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/tmp102.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/tmp103.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/tmp401.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/tmp421.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/via-cputemp.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/via686a.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/vt1211.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/vt8231.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83627ehf.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83627hf.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83781d.ko 2024-08-17T02:34:39.301 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83791d.ko 2024-08-17T02:34:39.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83792d.ko 2024-08-17T02:34:39.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83793.ko 2024-08-17T02:34:39.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83795.ko 2024-08-17T02:34:39.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83l785ts.ko 2024-08-17T02:34:39.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/hwmon/w83l786ng.ko 2024-08-17T02:34:39.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c 2024-08-17T02:34:39.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/algos 2024-08-17T02:34:39.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-08-17T02:34:39.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-08-17T02:34:39.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses 2024-08-17T02:34:39.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-08-17T02:34:39.302 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-i801.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-isch.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-parport.ko 2024-08-17T02:34:39.303 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-08-17T02:34:39.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-08-17T02:34:39.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-08-17T02:34:39.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-08-17T02:34:39.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-08-17T02:34:39.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-08-17T02:34:39.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-via.ko 2024-08-17T02:34:39.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-08-17T02:34:39.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-08-17T02:34:39.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-dev.ko 2024-08-17T02:34:39.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-mux.ko 2024-08-17T02:34:39.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-08-17T02:34:39.304 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-smbus.ko 2024-08-17T02:34:39.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/i2c/i2c-stub.ko 2024-08-17T02:34:39.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband 2024-08-17T02:34:39.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core 2024-08-17T02:34:39.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/ib_cm.ko 2024-08-17T02:34:39.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/ib_core.ko 2024-08-17T02:34:39.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/ib_umad.ko 2024-08-17T02:34:39.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-08-17T02:34:39.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/iw_cm.ko 2024-08-17T02:34:39.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/rdma_cm.ko 2024-08-17T02:34:39.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-08-17T02:34:39.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw 2024-08-17T02:34:39.305 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/cxgb4 2024-08-17T02:34:39.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-08-17T02:34:39.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mlx4 2024-08-17T02:34:39.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-08-17T02:34:39.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mlx5 2024-08-17T02:34:39.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-08-17T02:34:39.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mthca 2024-08-17T02:34:39.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-08-17T02:34:39.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp 2024-08-17T02:34:39.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/ipoib 2024-08-17T02:34:39.306 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/iser 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/isert 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/srp 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/srpt 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/gameport 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/gameport/fm801-gp.ko 2024-08-17T02:34:39.307 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/gameport/gameport.ko 2024-08-17T02:34:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/joydev.ko 2024-08-17T02:34:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/keyboard 2024-08-17T02:34:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/keyboard/gpio_keys.ko 2024-08-17T02:34:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse 2024-08-17T02:34:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/appletouch.ko 2024-08-17T02:34:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/bcm5974.ko 2024-08-17T02:34:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/cyapatp.ko 2024-08-17T02:34:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/elan_i2c.ko 2024-08-17T02:34:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/sermouse.ko 2024-08-17T02:34:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-08-17T02:34:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/synaptics_usb.ko 2024-08-17T02:34:39.308 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/mouse/vsxxxaa.ko 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/rmi4 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/rmi4/rmi_core.ko 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio/altera_ps2.ko 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio/arc_ps2.ko 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/serio/serio_raw.ko 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/input/sparse-keymap.ko 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/leds-blinkm.ko 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/leds-lm3530.ko 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/leds-lp3944.ko 2024-08-17T02:34:39.309 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/leds-ss4200.ko 2024-08-17T02:34:39.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger 2024-08-17T02:34:39.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-08-17T02:34:39.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-08-17T02:34:39.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-08-17T02:34:39.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-08-17T02:34:39.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-08-17T02:34:39.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-08-17T02:34:39.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-08-17T02:34:39.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-08-17T02:34:39.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md 2024-08-17T02:34:39.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/bcache 2024-08-17T02:34:39.310 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/bcache/bcache.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-bio-prison.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-cache-smq.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-cache.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-crypt.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-delay.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-flakey.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-log-userspace.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-multipath.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-queue-length.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-raid.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-round-robin.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-service-time.ko 2024-08-17T02:34:39.311 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-switch.ko 2024-08-17T02:34:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-thin-pool.ko 2024-08-17T02:34:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/dm-verity.ko 2024-08-17T02:34:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/persistent-data 2024-08-17T02:34:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-08-17T02:34:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/raid0.ko 2024-08-17T02:34:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/raid1.ko 2024-08-17T02:34:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/raid10.ko 2024-08-17T02:34:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/md/raid456.ko 2024-08-17T02:34:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message 2024-08-17T02:34:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion 2024-08-17T02:34:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptbase.ko 2024-08-17T02:34:39.312 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptctl.ko 2024-08-17T02:34:39.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptfc.ko 2024-08-17T02:34:39.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptlan.ko 2024-08-17T02:34:39.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptsas.ko 2024-08-17T02:34:39.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptscsih.ko 2024-08-17T02:34:39.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/message/fusion/mptspi.ko 2024-08-17T02:34:39.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd 2024-08-17T02:34:39.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/lpc_ich.ko 2024-08-17T02:34:39.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/lpc_sch.ko 2024-08-17T02:34:39.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/mfd-core.ko 2024-08-17T02:34:39.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/sm501.ko 2024-08-17T02:34:39.313 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/viperboard.ko 2024-08-17T02:34:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/vx855.ko 2024-08-17T02:34:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/mfd/wl1273-core.ko 2024-08-17T02:34:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc 2024-08-17T02:34:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/altera-stapl 2024-08-17T02:34:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-08-17T02:34:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/apds9802als.ko 2024-08-17T02:34:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/apds990x.ko 2024-08-17T02:34:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/bh1770glc.ko 2024-08-17T02:34:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/cb710 2024-08-17T02:34:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/cb710/cb710.ko 2024-08-17T02:34:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/eeprom 2024-08-17T02:34:39.314 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/eeprom/at24.ko 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/eeprom/max6875.ko 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/enclosure.ko 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/hpilo.ko 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/isl29003.ko 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/isl29020.ko 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/lis3lv02d 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/mei 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/mei/mei-me.ko 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/mei/mei-txe.ko 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/mei/mei.ko 2024-08-17T02:34:39.315 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-gru 2024-08-17T02:34:39.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-gru/gru.ko 2024-08-17T02:34:39.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-xp 2024-08-17T02:34:39.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-xp/xp.ko 2024-08-17T02:34:39.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-xp/xpc.ko 2024-08-17T02:34:39.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-08-17T02:34:39.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/tifm_7xx1.ko 2024-08-17T02:34:39.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/tifm_core.ko 2024-08-17T02:34:39.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/tsl2550.ko 2024-08-17T02:34:39.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/vmw_balloon.ko 2024-08-17T02:34:39.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/vmw_vmci 2024-08-17T02:34:39.316 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/bonding 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/bonding/bonding.ko 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/b53 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/b53/b53_common.ko 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/bcm-sf2.ko 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/mv88e6060.ko 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/mv88e6xxx 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/dummy.ko 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/eql.ko 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com 2024-08-17T02:34:39.317 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-08-17T02:34:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-08-17T02:34:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-08-17T02:34:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-08-17T02:34:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390 2024-08-17T02:34:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390/8390.ko 2024-08-17T02:34:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-08-17T02:34:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-08-17T02:34:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-08-17T02:34:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/adaptec 2024-08-17T02:34:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-08-17T02:34:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/agere 2024-08-17T02:34:39.318 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/agere/et131x.ko 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/alteon 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/altera 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/amd 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/alx 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atl1c 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-08-17T02:34:39.319 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atl1e 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atlx 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/genet 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-08-17T02:34:39.320 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/brocade 2024-08-17T02:34:39.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/brocade/bna 2024-08-17T02:34:39.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-08-17T02:34:39.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio 2024-08-17T02:34:39.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb 2024-08-17T02:34:39.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-08-17T02:34:39.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-08-17T02:34:39.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-08-17T02:34:39.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-08-17T02:34:39.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-08-17T02:34:39.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-08-17T02:34:39.321 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/cisco 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/cisco/enic 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-08-17T02:34:39.322 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-08-17T02:34:39.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dlink 2024-08-17T02:34:39.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-08-17T02:34:39.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-08-17T02:34:39.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/dnet.ko 2024-08-17T02:34:39.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/emulex 2024-08-17T02:34:39.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/emulex/benet 2024-08-17T02:34:39.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-08-17T02:34:39.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/ethoc.ko 2024-08-17T02:34:39.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/fealnx.ko 2024-08-17T02:34:39.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel 2024-08-17T02:34:39.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e100.ko 2024-08-17T02:34:39.323 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e1000 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e1000e 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/fm10k 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/i40e 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/iavf 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/igb 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/igbvf 2024-08-17T02:34:39.324 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/ixgbe 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/ixgbevf 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/libeth 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/libie 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/jme.ko 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/marvell 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/marvell/skge.ko 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-08-17T02:34:39.325 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx4 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx5 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/micrel 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/myricom 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/myricom/myri10ge 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/natsemi 2024-08-17T02:34:39.326 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/neterion 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/nvidia 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/packetengines 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/netxen 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-08-17T02:34:39.327 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/rdc 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek/atp.ko 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/rocker 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sfc 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/silan 2024-08-17T02:34:39.328 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-08-17T02:34:39.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sis 2024-08-17T02:34:39.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sis/sis190.ko 2024-08-17T02:34:39.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sis/sis900.ko 2024-08-17T02:34:39.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc 2024-08-17T02:34:39.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-08-17T02:34:39.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-08-17T02:34:39.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-08-17T02:34:39.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-08-17T02:34:39.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/stmicro 2024-08-17T02:34:39.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/stmicro/stmmac 2024-08-17T02:34:39.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-08-17T02:34:39.329 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-08-17T02:34:39.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun 2024-08-17T02:34:39.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun/cassini.ko 2024-08-17T02:34:39.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun/niu.ko 2024-08-17T02:34:39.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun/sungem.ko 2024-08-17T02:34:39.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-08-17T02:34:39.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/tehuti 2024-08-17T02:34:39.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-08-17T02:34:39.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/ti 2024-08-17T02:34:39.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/ti/tlan.ko 2024-08-17T02:34:39.330 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/via 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/wiznet 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/xircom 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/geneve.ko 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/hyperv 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ieee802154 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ieee802154/fakelb.ko 2024-08-17T02:34:39.331 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ifb.ko 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ipvlan 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ipvlan/ipvlan.ko 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/macvlan.ko 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/macvtap.ko 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio.ko 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mdio/mdio-regmap.ko 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/mii.ko 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/net_failover.ko 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/netconsole.ko 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/nlmon.ko 2024-08-17T02:34:39.332 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/pcs 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/pcs/pcs-lynx.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/amd.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/bcm7xxx.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/bcm87xx.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/broadcom.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/cicada.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/davicom.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/dp83640.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/et1011c.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/icplus.ko 2024-08-17T02:34:39.333 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/lxt.ko 2024-08-17T02:34:39.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/marvell.ko 2024-08-17T02:34:39.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/micrel.ko 2024-08-17T02:34:39.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/national.ko 2024-08-17T02:34:39.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/phylink.ko 2024-08-17T02:34:39.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/qsemi.ko 2024-08-17T02:34:39.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/realtek.ko 2024-08-17T02:34:39.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/smsc.ko 2024-08-17T02:34:39.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/ste10Xp.ko 2024-08-17T02:34:39.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/phy/vitesse.ko 2024-08-17T02:34:39.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp 2024-08-17T02:34:39.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/bsd_comp.ko 2024-08-17T02:34:39.334 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_async.ko 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_deflate.ko 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_generic.ko 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_mppe.ko 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/ppp_synctty.ko 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/pppoe.ko 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/pppox.ko 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/ppp/pptp.ko 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/slip 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/slip/slhc.ko 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/slip/slip.ko 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/sungem_phy.ko 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/tap.ko 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team 2024-08-17T02:34:39.335 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team.ko 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_activebackup.ko 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_broadcast.ko 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_random.ko 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/tun.ko 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/veth.ko 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/virtio_net.ko 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/vmxnet3 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/vxlan 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/vxlan/vxlan.ko 2024-08-17T02:34:39.336 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/xen-netback 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/xen-netback/xen-netback.ko 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/net/xen-netfront.ko 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ntb 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ntb/ntb.ko 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/nvme 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/nvme/host 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/nvme/host/nvme-core.ko 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/nvme/host/nvme.ko 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport/parport.ko 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport/parport_cs.ko 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport/parport_pc.ko 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/parport/parport_serial.ko 2024-08-17T02:34:39.337 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/hotplug 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/pcie 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/pcie/aer_inject.ko 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pci/xen-pcifront.ko 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pcmcia 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pcmcia/i82092.ko 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pcmcia/pd6729.ko 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pcmcia/yenta_socket.ko 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pinctrl 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pinctrl/intel 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-08-17T02:34:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/acer-wmi.ko 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/acerhdf.ko 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/apple-gmux.ko 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/asus-laptop.ko 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/asus-wmi.ko 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/classmate-laptop.ko 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-08-17T02:34:39.339 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/hdaps.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/intel 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/intel_ips.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/msi-wmi.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/mxm-wmi.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/samsung-laptop.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/samsung-q10.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/topstar-laptop.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/toshiba_haps.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/wmi-bmof.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/platform/x86/wmi.ko 2024-08-17T02:34:39.340 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/powercap 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/powercap/intel_rapl_common.ko 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/powercap/intel_rapl_msr.ko 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/clients 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/clients/pps-gpio.ko 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/clients/pps-ldisc.ko 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/clients/pps_parport.ko 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/pps/pps_core.ko 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ptp 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ptp/ptp.ko 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ptp/ptp_kvm.ko 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras/amd 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras/amd/atl 2024-08-17T02:34:39.341 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-08-17T02:34:39.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ras/amd/fmpm.ko 2024-08-17T02:34:39.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc 2024-08-17T02:34:39.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-bq32k.ko 2024-08-17T02:34:39.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1286.ko 2024-08-17T02:34:39.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1307.ko 2024-08-17T02:34:39.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1374.ko 2024-08-17T02:34:39.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1511.ko 2024-08-17T02:34:39.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1553.ko 2024-08-17T02:34:39.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1672.ko 2024-08-17T02:34:39.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds1742.ko 2024-08-17T02:34:39.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds2404.ko 2024-08-17T02:34:39.342 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-ds3232.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-em3027.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-fm3130.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-isl12022.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-isl1208.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-m41t80.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-m48t35.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-m48t59.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-max6900.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-msm6242.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf2127.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf85063.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf8523.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf8563.ko 2024-08-17T02:34:39.343 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-pcf8583.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rp5c01.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rs5c372.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rx8025.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-rx8581.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/rtc/rtc-x1205.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/3w-9xxx.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/3w-sas.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/3w-xxxx.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/BusLogic.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/a100u2w.ko 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aacraid 2024-08-17T02:34:39.344 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aacraid/aacraid.ko 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/advansys.ko 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aic7xxx 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/am53c974.ko 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/arcmsr 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/atp870u.ko 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/be2iscsi 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bfa 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bfa/bfa.ko 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bnx2fc 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-08-17T02:34:39.345 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bnx2i 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/ch.ko 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/csiostor 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/csiostor/csiostor.ko 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/cxgb3i 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/cxgb4i 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/dc395x.ko 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-08-17T02:34:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-08-17T02:34:39.347 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-08-17T02:34:39.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-08-17T02:34:39.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/dmx3191d.ko 2024-08-17T02:34:39.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/esas2r 2024-08-17T02:34:39.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/esas2r/esas2r.ko 2024-08-17T02:34:39.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/esp_scsi.ko 2024-08-17T02:34:39.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fcoe 2024-08-17T02:34:39.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fcoe/fcoe.ko 2024-08-17T02:34:39.349 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fnic 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/fnic/fnic.ko 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/hpsa.ko 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/hptiop.ko 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/hv_storvsc.ko 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/initio.ko 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/ipr.ko 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/ips.ko 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/isci 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/isci/isci.ko 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/iscsi_tcp.ko 2024-08-17T02:34:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libfc 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libfc/libfc.ko 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libiscsi.ko 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libiscsi_tcp.ko 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libsas 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/libsas/libsas.ko 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/lpfc 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/lpfc/lpfc.ko 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid.ko 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-08-17T02:34:39.351 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mpt3sas 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mvsas 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mvsas/mvsas.ko 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/mvumi.ko 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/pm8001 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/pmcraid.ko 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla1280.ko 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla2xxx 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla4xxx 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/raid_class.ko 2024-08-17T02:34:39.352 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_debug.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_fc.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_sas.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_spi.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/scsi_transport_srp.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/ses.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/st.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/stex.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/sym53c8xx_2 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/virtio_scsi.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/vmw_pvscsi.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/scsi/wd719x.ko 2024-08-17T02:34:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ssb 2024-08-17T02:34:39.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ssb/ssb.ko 2024-08-17T02:34:39.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target 2024-08-17T02:34:39.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/iscsi 2024-08-17T02:34:39.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-08-17T02:34:39.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/loopback 2024-08-17T02:34:39.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/loopback/tcm_loop.ko 2024-08-17T02:34:39.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_file.ko 2024-08-17T02:34:39.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_iblock.ko 2024-08-17T02:34:39.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_mod.ko 2024-08-17T02:34:39.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_pscsi.ko 2024-08-17T02:34:39.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/target_core_user.ko 2024-08-17T02:34:39.354 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/tcm_fc 2024-08-17T02:34:39.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-08-17T02:34:39.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal 2024-08-17T02:34:39.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel 2024-08-17T02:34:39.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal 2024-08-17T02:34:39.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-08-17T02:34:39.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-08-17T02:34:39.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-08-17T02:34:39.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-08-17T02:34:39.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-08-17T02:34:39.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-08-17T02:34:39.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-08-17T02:34:39.355 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/ipwireless 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-08-17T02:34:39.356 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/n_gsm.ko 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/n_hdlc.ko 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/nozomi.ko 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/8250 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/arc_uart.ko 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/jsm 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/tty/serial/jsm/jsm.ko 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs/core 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs/core/ufshcd-core.ko 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs/host 2024-08-17T02:34:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-08-17T02:34:39.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio 2024-08-17T02:34:39.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio.ko 2024-08-17T02:34:39.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio_aec.ko 2024-08-17T02:34:39.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio_cif.ko 2024-08-17T02:34:39.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio_pci_generic.ko 2024-08-17T02:34:39.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/uio/uio_sercos3.ko 2024-08-17T02:34:39.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb 2024-08-17T02:34:39.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class 2024-08-17T02:34:39.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class/cdc-acm.ko 2024-08-17T02:34:39.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class/cdc-wdm.ko 2024-08-17T02:34:39.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class/usblp.ko 2024-08-17T02:34:39.358 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/class/usbtmc.ko 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/host 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/host/sl811-hcd.ko 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/image 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/image/mdc800.ko 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/image/microtek.ko 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/adutux.ko 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/appledisplay.ko 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/emi26.ko 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/emi62.ko 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/ezusb.ko 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/idmouse.ko 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/iowarrior.ko 2024-08-17T02:34:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/isight_firmware.ko 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/ldusb.ko 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/legousbtower.ko 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/sisusbvga 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/trancevibrator.ko 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/usb3503.ko 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/usblcd.ko 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/usbsevseg.ko 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/uss720.ko 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/misc/yurex.ko 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/phy 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/phy/phy-generic.ko 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/aircable.ko 2024-08-17T02:34:39.360 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ark3116.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/belkin_sa.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ch341.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/cp210x.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/cyberjack.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/cypress_m8.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/digi_acceleport.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/empeg.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ftdi_sio.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/garmin_gps.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/io_edgeport.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/io_ti.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ipaq.ko 2024-08-17T02:34:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ipw.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ir-usb.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/keyspan.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/keyspan_pda.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/kl5kusb105.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/kobil_sct.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/mct_u232.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/mos7720.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/mos7840.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/navman.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/omninet.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/opticon.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/option.ko 2024-08-17T02:34:39.362 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/oti6858.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/pl2303.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/qcaux.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/qcserial.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/quatech2.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/safe_serial.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/sierra.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/spcp8x5.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ssu100.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/symbolserial.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/usb_debug.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/usb_wwan.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/visor.ko 2024-08-17T02:34:39.363 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/whiteheat.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/serial/xsens_mt.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/uas.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-alauda.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-cypress.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-datafab.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-freecom.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-isd200.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-karma.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-onetouch.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-realtek.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-sddr09.ko 2024-08-17T02:34:39.364 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-sddr55.ko 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/ums-usbat.ko 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/storage/usb-storage.ko 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/usbip 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/usbip/usbip-core.ko 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/usbip/usbip-host.ko 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/pci 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/pci/vfio-pci.ko 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/vfio.ko 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost 2024-08-17T02:34:39.365 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost/vhost.ko 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost/vhost_iotlb.ko 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost/vhost_net.ko 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/vhost/vhost_scsi.ko 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video/backlight 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video/backlight/apple_bl.ko 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video/backlight/lcd.ko 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/video/backlight/platform_lcd.ko 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio.ko 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_balloon.ko 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_mmio.ko 2024-08-17T02:34:39.366 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_pci.ko 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/virtio/virtio_ring.ko 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/masters 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/masters/ds2482.ko 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/masters/ds2490.ko 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-08-17T02:34:39.367 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-08-17T02:34:39.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-08-17T02:34:39.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-08-17T02:34:39.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-08-17T02:34:39.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-08-17T02:34:39.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_smem.ko 2024-08-17T02:34:39.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/slaves/w1_therm.ko 2024-08-17T02:34:39.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/w1/wire.ko 2024-08-17T02:34:39.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog 2024-08-17T02:34:39.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/alim1535_wdt.ko 2024-08-17T02:34:39.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/alim7101_wdt.ko 2024-08-17T02:34:39.368 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/f71808e_wdt.ko 2024-08-17T02:34:39.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/hpwdt.ko 2024-08-17T02:34:39.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/i6300esb.ko 2024-08-17T02:34:39.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-08-17T02:34:39.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/iTCO_wdt.ko 2024-08-17T02:34:39.369 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/ib700wdt.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/ibmasr.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/it8712f_wdt.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/it87_wdt.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/machzwd.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/nv_tco.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/pcwd_pci.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/pcwd_usb.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/sch311x_wdt.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/softdog.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/sp5100_tco.ko 2024-08-17T02:34:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/via_wdt.ko 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/w83877f_wdt.ko 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/w83977f_wdt.ko 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/wdt_pci.ko 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/watchdog/xen_wdt.ko 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-acpi-processor.ko 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-evtchn.ko 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-gntalloc.ko 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-gntdev.ko 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-pciback 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-08-17T02:34:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xen-privcmd.ko 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xenfs 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/drivers/xen/xenfs/xenfs.ko 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/9p 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/9p/9p.ko 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/affs 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/affs/affs.ko 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/befs 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/befs/befs.ko 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/binfmt_misc.ko 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/btrfs 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/btrfs/btrfs.ko 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/cachefiles 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/cachefiles/cachefiles.ko 2024-08-17T02:34:39.372 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ceph 2024-08-17T02:34:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ceph/ceph.ko 2024-08-17T02:34:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/coda 2024-08-17T02:34:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/coda/coda.ko 2024-08-17T02:34:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/cramfs 2024-08-17T02:34:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/cramfs/cramfs.ko 2024-08-17T02:34:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/dlm 2024-08-17T02:34:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/dlm/dlm.ko 2024-08-17T02:34:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ecryptfs 2024-08-17T02:34:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ecryptfs/ecryptfs.ko 2024-08-17T02:34:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/f2fs 2024-08-17T02:34:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/f2fs/f2fs.ko 2024-08-17T02:34:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fat 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fat/fat.ko 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fat/msdos.ko 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fat/vfat.ko 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fuse 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fuse/cuse.ko 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/fuse/fuse.ko 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/gfs2 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/gfs2/gfs2.ko 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/hfs 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/hfs/hfs.ko 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/hfsplus 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/hfsplus/hfsplus.ko 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/isofs 2024-08-17T02:34:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/isofs/isofs.ko 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/jfs 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/jfs/jfs.ko 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/lockd 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/lockd/lockd.ko 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/minix 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/minix/minix.ko 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/netfs 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/netfs/netfs.ko 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/blocklayout 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/filelayout 2024-08-17T02:34:39.375 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/flexfilelayout 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/nfs.ko 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/nfsv3.ko 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs/nfsv4.ko 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs_common 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs_common/grace.ko 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfs_common/nfs_acl.ko 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfsd 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nfsd/nfsd.ko 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nilfs2 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nilfs2/nilfs2.ko 2024-08-17T02:34:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls 2024-08-17T02:34:39.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-celtic.ko 2024-08-17T02:34:39.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-centeuro.ko 2024-08-17T02:34:39.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-croatian.ko 2024-08-17T02:34:39.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-cyrillic.ko 2024-08-17T02:34:39.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-gaelic.ko 2024-08-17T02:34:39.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-greek.ko 2024-08-17T02:34:39.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-iceland.ko 2024-08-17T02:34:39.377 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-inuit.ko 2024-08-17T02:34:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-roman.ko 2024-08-17T02:34:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-romanian.ko 2024-08-17T02:34:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/mac-turkish.ko 2024-08-17T02:34:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp1250.ko 2024-08-17T02:34:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp1251.ko 2024-08-17T02:34:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp1255.ko 2024-08-17T02:34:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp737.ko 2024-08-17T02:34:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp775.ko 2024-08-17T02:34:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp850.ko 2024-08-17T02:34:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp852.ko 2024-08-17T02:34:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp855.ko 2024-08-17T02:34:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp857.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp860.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp861.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp862.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp863.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp864.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp865.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp866.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp869.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp874.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp932.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp936.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp949.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_cp950.ko 2024-08-17T02:34:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_euc-jp.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-1.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-13.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-14.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-15.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-2.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-3.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-4.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-5.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-6.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-7.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_iso8859-9.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_koi8-r.ko 2024-08-17T02:34:39.380 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_koi8-ru.ko 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_koi8-u.ko 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_ucs2_utils.ko 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/nls/nls_utf8.ko 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/cluster 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/dlm 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/dlmfs 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/ocfs2.ko 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-08-17T02:34:39.381 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-08-17T02:34:39.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/overlayfs 2024-08-17T02:34:39.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/overlayfs/overlay.ko 2024-08-17T02:34:39.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/pstore 2024-08-17T02:34:39.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/pstore/ramoops.ko 2024-08-17T02:34:39.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/reiserfs 2024-08-17T02:34:39.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/reiserfs/reiserfs.ko 2024-08-17T02:34:39.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/romfs 2024-08-17T02:34:39.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/romfs/romfs.ko 2024-08-17T02:34:39.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb 2024-08-17T02:34:39.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/client 2024-08-17T02:34:39.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/client/cifs.ko 2024-08-17T02:34:39.382 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/common 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/common/cifs_arc4.ko 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/smb/common/cifs_md4.ko 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/squashfs 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/squashfs/squashfs.ko 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/sysv 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/sysv/sysv.ko 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/udf 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/udf/udf.ko 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ufs 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/ufs/ufs.ko 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/xfs 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/fs/xfs/xfs.ko 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel 2024-08-17T02:34:39.383 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/rcu 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/rcu/rcutorture.ko 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/torture.ko 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/trace 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/asn1_encoder.ko 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crc-itu-t.ko 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crc8.ko 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crypto 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crypto/libaescfb.ko 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crypto/libarc4.ko 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/crypto/libdes.ko 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/libcrc32c.ko 2024-08-17T02:34:39.384 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/lru_cache.ko 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/lz4 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/lz4/lz4_compress.ko 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/lz4/lz4hc_compress.ko 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/math 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/math/cordic.ko 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/raid6 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/raid6/raid6_pq.ko 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/reed_solomon 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/reed_solomon/reed_solomon.ko 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/ts_bm.ko 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/ts_fsm.ko 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/ts_kmp.ko 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/zstd 2024-08-17T02:34:39.385 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/lib/zstd/zstd_compress.ko 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/mm 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/mm/hwpoison-inject.ko 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/6lowpan.ko 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_dest.ko 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_fragment.ko 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_hop.ko 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_ipv6.ko 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_mobility.ko 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_routing.ko 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/6lowpan/nhc_udp.ko 2024-08-17T02:34:39.386 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802 2024-08-17T02:34:39.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/garp.ko 2024-08-17T02:34:39.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/mrp.ko 2024-08-17T02:34:39.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/p8022.ko 2024-08-17T02:34:39.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/psnap.ko 2024-08-17T02:34:39.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/802/stp.ko 2024-08-17T02:34:39.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/8021q 2024-08-17T02:34:39.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/8021q/8021q.ko 2024-08-17T02:34:39.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p 2024-08-17T02:34:39.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p/9pnet.ko 2024-08-17T02:34:39.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p/9pnet_fd.ko 2024-08-17T02:34:39.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p/9pnet_rdma.ko 2024-08-17T02:34:39.387 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/9p/9pnet_virtio.ko 2024-08-17T02:34:39.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/batman-adv 2024-08-17T02:34:39.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/batman-adv/batman-adv.ko 2024-08-17T02:34:39.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge 2024-08-17T02:34:39.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/br_netfilter.ko 2024-08-17T02:34:39.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/bridge.ko 2024-08-17T02:34:39.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter 2024-08-17T02:34:39.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-08-17T02:34:39.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_among.ko 2024-08-17T02:34:39.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_arp.ko 2024-08-17T02:34:39.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-08-17T02:34:39.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-08-17T02:34:39.388 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_ip.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_limit.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_log.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_mark.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_snat.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_stp.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-08-17T02:34:39.389 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/ebtables.ko 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ceph 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ceph/libceph.ko 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/core 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/core/failover.ko 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/core/pktgen.ko 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp/dccp.ko 2024-08-17T02:34:39.390 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp/dccp_diag.ko 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp/dccp_ipv4.ko 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dccp/dccp_ipv6.ko 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dns_resolver 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dns_resolver/dns_resolver.ko 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/dsa_core.ko 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/tag_brcm.ko 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/tag_dsa.ko 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/tag_none.ko 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/dsa/tag_trailer.ko 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154/6lowpan 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-08-17T02:34:39.391 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154/ieee802154.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ieee802154/ieee802154_socket.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ah4.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/esp4.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/fou.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/gre.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/inet_diag.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ip_gre.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ip_tunnel.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ip_vti.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ipcomp.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/ipip.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/arp_tables.ko 2024-08-17T02:34:39.392 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/iptable_security.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-08-17T02:34:39.393 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-08-17T02:34:39.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-08-17T02:34:39.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-08-17T02:34:39.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-08-17T02:34:39.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-08-17T02:34:39.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_bic.ko 2024-08-17T02:34:39.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_dctcp.ko 2024-08-17T02:34:39.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_diag.ko 2024-08-17T02:34:39.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_highspeed.ko 2024-08-17T02:34:39.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_htcp.ko 2024-08-17T02:34:39.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_hybla.ko 2024-08-17T02:34:39.394 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_illinois.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_lp.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_scalable.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_vegas.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_veno.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_westwood.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tcp_yeah.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/tunnel4.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/udp_diag.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/udp_tunnel.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv4/xfrm4_tunnel.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ah6.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/esp6.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/fou6.ko 2024-08-17T02:34:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ip6_tunnel.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ip6_vti.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/ipcomp6.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-08-17T02:34:39.396 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-08-17T02:34:39.397 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/sit.ko 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/tunnel6.ko 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/ipv6/xfrm6_tunnel.ko 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/key 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/key/af_key.ko 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_core.ko 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_debugfs.ko 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_eth.ko 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_ip.ko 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_ip6.ko 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_netlink.ko 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/l2tp/l2tp_ppp.ko 2024-08-17T02:34:39.398 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/llc 2024-08-17T02:34:39.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/llc/llc.ko 2024-08-17T02:34:39.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/mac802154 2024-08-17T02:34:39.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/mac802154/mac802154.ko 2024-08-17T02:34:39.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/mpls 2024-08-17T02:34:39.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/mpls/mpls_gso.ko 2024-08-17T02:34:39.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter 2024-08-17T02:34:39.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset 2024-08-17T02:34:39.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set.ko 2024-08-17T02:34:39.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-08-17T02:34:39.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-08-17T02:34:39.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-08-17T02:34:39.399 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs 2024-08-17T02:34:39.400 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conncount.ko 2024-08-17T02:34:39.401 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_h323.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_irc.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_sane.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_sip.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_dup_netdev.ko 2024-08-17T02:34:39.402 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_flow_table.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_flow_table_inet.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_log_syslog.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_amanda.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_ftp.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_irc.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_sip.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_nat_tftp.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_synproxy_core.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nf_tables.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink_acct.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink_log.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink_osf.ko 2024-08-17T02:34:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nfnetlink_queue.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_chain_nat.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_compat.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_ct.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_dup_netdev.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_fib.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_fwd_netdev.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_hash.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_limit.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_log.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_masq.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_nat.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_numgen.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_queue.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_quota.ko 2024-08-17T02:34:39.404 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_redir.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_reject.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_reject_inet.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_socket.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/nft_tunnel.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_AUDIT.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_CHECKSUM.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_CLASSIFY.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_CT.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_DSCP.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_HL.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_HMARK.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_IDLETIMER.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_LED.ko 2024-08-17T02:34:39.405 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_LOG.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_MASQUERADE.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_NETMAP.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_NFLOG.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_NFQUEUE.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_RATEEST.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_REDIRECT.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_SECMARK.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TCPMSS.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TEE.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TPROXY.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_TRACE.ko 2024-08-17T02:34:39.406 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_addrtype.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_bpf.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_cgroup.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_cluster.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_comment.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_connbytes.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_connlabel.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_connlimit.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_connmark.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_conntrack.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_cpu.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_dccp.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_devgroup.ko 2024-08-17T02:34:39.407 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_dscp.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_ecn.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_esp.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_hashlimit.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_helper.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_hl.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_ipcomp.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_iprange.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_ipvs.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_l2tp.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_length.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_limit.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_mac.ko 2024-08-17T02:34:39.408 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_mark.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_multiport.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_nat.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_nfacct.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_osf.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_owner.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_physdev.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_pkttype.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_policy.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_quota.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_rateest.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_realm.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_recent.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_sctp.ko 2024-08-17T02:34:39.409 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_set.ko 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_socket.ko 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_state.ko 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_statistic.ko 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_string.ko 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_tcpmss.ko 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_time.ko 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netfilter/xt_u32.ko 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netlink 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/netlink/netlink_diag.ko 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/nsh 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/nsh/nsh.ko 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch 2024-08-17T02:34:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch/openvswitch.ko 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch/vport-geneve.ko 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch/vport-gre.ko 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/openvswitch/vport-vxlan.ko 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/packet 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/packet/af_packet_diag.ko 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/rds 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/rds/rds.ko 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/rds/rds_rdma.ko 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/rds/rds_tcp.ko 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_csum.ko 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_gact.ko 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_mirred.ko 2024-08-17T02:34:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_nat.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_pedit.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_police.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_simple.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_skbedit.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/act_vlan.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_basic.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_bpf.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_flow.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_fw.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_route.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/cls_u32.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_cmp.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_ipset.ko 2024-08-17T02:34:39.412 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_meta.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_nbyte.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_text.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/em_u32.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_choke.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_codel.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_drr.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_fq.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_gred.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_hfsc.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_hhf.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_htb.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_ingress.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_mqprio.ko 2024-08-17T02:34:39.413 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_mqprio_lib.ko 2024-08-17T02:34:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_multiq.ko 2024-08-17T02:34:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_netem.ko 2024-08-17T02:34:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_pie.ko 2024-08-17T02:34:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_plug.ko 2024-08-17T02:34:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_prio.ko 2024-08-17T02:34:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_qfq.ko 2024-08-17T02:34:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_red.ko 2024-08-17T02:34:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_sfb.ko 2024-08-17T02:34:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_sfq.ko 2024-08-17T02:34:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_tbf.ko 2024-08-17T02:34:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sched/sch_teql.ko 2024-08-17T02:34:39.414 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sctp 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sctp/sctp.ko 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sctp/sctp_diag.ko 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/auth_gss 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/sunrpc.ko 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/xprtrdma 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/tipc 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/tipc/tipc.ko 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/tipc/tipc_diag.ko 2024-08-17T02:34:39.415 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/unix 2024-08-17T02:34:39.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/unix/unix_diag.ko 2024-08-17T02:34:39.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock 2024-08-17T02:34:39.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-08-17T02:34:39.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-08-17T02:34:39.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vsock.ko 2024-08-17T02:34:39.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vsock_diag.ko 2024-08-17T02:34:39.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/vmw_vsock/vsock_loopback.ko 2024-08-17T02:34:39.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/xfrm 2024-08-17T02:34:39.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/net/xfrm/xfrm_ipcomp.ko 2024-08-17T02:34:39.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security 2024-08-17T02:34:39.416 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys/encrypted-keys 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys/trusted-keys 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/kernel/security/keys/trusted-keys/trusted.ko 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.alias 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.alias.bin 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.builtin 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.builtin.alias.bin 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.builtin.bin 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.builtin.modinfo 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.dep 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.dep.bin 2024-08-17T02:34:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.devname 2024-08-17T02:34:39.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.order 2024-08-17T02:34:39.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.softdep 2024-08-17T02:34:39.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.symbols 2024-08-17T02:34:39.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.symbols.bin 2024-08-17T02:34:39.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.weakdep 2024-08-17T02:34:39.418 INFO:teuthology.orchestra.run.smithi016.stdout:/lib/modules/6.11.0-rc3-g7503b862ff5b/vmlinuz 2024-08-17T02:34:39.421 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:39.421 DEBUG:teuthology.orchestra.run.smithi016:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-08-17T02:34:39.444 INFO:teuthology.orchestra.run.smithi016.stdout:postinstall scriptlet (using /bin/sh): 2024-08-17T02:34:39.444 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-17T02:34:39.444 INFO:teuthology.orchestra.run.smithi016.stdout: /usr/bin/kernel-install add 6.11.0-rc3-g7503b862ff5b /lib/modules/6.11.0-rc3-g7503b862ff5b/vmlinuz 2024-08-17T02:34:39.444 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-08-17T02:34:39.444 INFO:teuthology.orchestra.run.smithi016.stdout:for file in vmlinuz System.map config; do 2024-08-17T02:34:39.444 INFO:teuthology.orchestra.run.smithi016.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc3-g7503b862ff5b/${file}" "/boot/${file}-6.11.0-rc3-g7503b862ff5b"; then 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout: cp "/lib/modules/6.11.0-rc3-g7503b862ff5b/${file}" "/boot/${file}-6.11.0-rc3-g7503b862ff5b" 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout: fi 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout:done 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -d "/lib/modules/6.11.0-rc3-g7503b862ff5b/dtb" ] && \ 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout: ! diff -rq "/lib/modules/6.11.0-rc3-g7503b862ff5b/dtb" "/boot/dtb-6.11.0-rc3-g7503b862ff5b" >/dev/null 2>&1; then 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout: rm -rf "/boot/dtb-6.11.0-rc3-g7503b862ff5b" 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout: cp -r "/lib/modules/6.11.0-rc3-g7503b862ff5b/dtb" "/boot/dtb-6.11.0-rc3-g7503b862ff5b" 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout:if [ ! -e "/lib/modules/6.11.0-rc3-g7503b862ff5b/modules.dep" ]; then 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout: /usr/sbin/depmod 6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout:preuninstall scriptlet (using /bin/sh): 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-install remove 6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:39.445 INFO:teuthology.orchestra.run.smithi016.stdout:fi 2024-08-17T02:34:39.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 68.el9 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8264254 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-08-17T02:34:39.519 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-08-17T02:34:39.520 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-08-17T02:34:39.520 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-08-17T02:34:39.520 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-08-17T02:34:39.520 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-08-17T02:34:39.520 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-08-17T02:34:39.520 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:34:39.520 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-08-17T02:34:39.520 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-08-17T02:34:39.520 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-08-17T02:34:39.520 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-08-17T02:34:39.520 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:34:39.520 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-08-17T02:34:39.521 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-08-17T02:34:39.521 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:39.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-08-17T02:34:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-08-17T02:34:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-08-17T02:34:41.621 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-08-17T02:34:41.623 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-08-17T02:34:41.679 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.0Sv3lTxxJj 2024-08-17T02:34:41.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0Sv3lTxxJj 2024-08-17T02:34:41.752 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.0Sv3lTxxJj 2024-08-17T02:34:41.908 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.0Sv3lTxxJj is 6KB 2024-08-17T02:34:41.956 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.0Sv3lTxxJj 2024-08-17T02:34:41.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-08-17T02:34:42.032 INFO:teuthology.orchestra.run.smithi016.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-08-17T02:34:42.032 INFO:teuthology.orchestra.run.smithi016.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-08-17T02:34:42.032 INFO:teuthology.orchestra.run.smithi016.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-08-17T02:34:42.032 INFO:teuthology.orchestra.run.smithi016.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.11.0-rc3-g7503b862ff5b.conf 2024-08-17T02:34:42.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.11.0-rc3-g7503b862ff5b 2024-08-17T02:34:42.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-08-17T02:35:04.976 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-08-17T02:35:04.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-08-17T02:35:04.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:35:12.185 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-08-17T02:35:12.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-08-17T02:35:12.186 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:35:23.490 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-08-17T02:35:30.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2024-08-17T02:35:32.494 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-08-17T02:35:32.494 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:35:35.550 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-08-17T02:35:39.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-08-17T02:35:39.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:35:47.558 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-08-17T02:35:47.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:36:19.472 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-08-17T02:36:20.052 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-08-17T02:36:20.052 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc3-g7503b862ff5b"... 2024-08-17T02:36:20.052 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-08-17T02:36:20.086 INFO:teuthology.orchestra.run.smithi142.stdout:6.11.0-rc3-g7503b862ff5b 2024-08-17T02:36:20.086 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc3-g7503b862ff5b vs 6.11.0-rc3-g7503b862ff5b 2024-08-17T02:36:20.086 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-17T02:36:20.086 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-08-17T02:36:21.088 DEBUG:teuthology.parallel:result is None 2024-08-17T02:36:39.662 DEBUG:teuthology.orchestra.remote:timed out 2024-08-17T02:36:51.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-08-17T02:36:51.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:36:52.091 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-08-17T02:36:52.578 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-08-17T02:36:52.578 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc3-g7503b862ff5b"... 2024-08-17T02:36:52.578 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-08-17T02:36:52.614 INFO:teuthology.orchestra.run.smithi016.stdout:6.11.0-rc3-g7503b862ff5b 2024-08-17T02:36:52.614 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc3-g7503b862ff5b vs 6.11.0-rc3-g7503b862ff5b 2024-08-17T02:36:52.615 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-17T02:36:52.615 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-08-17T02:36:53.617 DEBUG:teuthology.parallel:result is None 2024-08-17T02:36:53.617 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-17T02:36:53.625 INFO:teuthology.task.internal:Creating test directory... 2024-08-17T02:36:53.625 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-17T02:36:53.629 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-17T02:36:53.671 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-17T02:36:53.711 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-17T02:36:53.719 INFO:teuthology.task.internal:Creating archive directory... 2024-08-17T02:36:53.719 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-17T02:36:53.723 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-17T02:36:53.778 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-17T02:36:53.785 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-17T02:36:53.785 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-17T02:36:53.789 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-17T02:36:53.876 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-17T02:36:53.916 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-17T02:36:53.938 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-17T02:36:53.979 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-17T02:36:53.982 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-17T02:36:53.989 INFO:teuthology.task.internal:Configuring sudo... 2024-08-17T02:36:53.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-17T02:36:53.993 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-17T02:36:54.050 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-17T02:36:54.060 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-17T02:36:54.060 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-17T02:36:54.064 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-17T02:36:54.099 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-17T02:36:54.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-17T02:36:54.277 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-17T02:36:54.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-17T02:36:54.410 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:36:54.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-17T02:36:54.465 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-17T02:36:54.530 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-17T02:36:54.607 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-17T02:36:54.668 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-17T02:36:54.739 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:36:54.739 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-17T02:36:54.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-08-17T02:36:54.797 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-08-17T02:36:54.871 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-17T02:36:54.919 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-17T02:36:55.398 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-17T02:36:55.406 INFO:teuthology.task.internal:Starting timer... 2024-08-17T02:36:55.406 INFO:teuthology.run_tasks:Running task pcp... 2024-08-17T02:36:55.417 INFO:teuthology.run_tasks:Running task selinux... 2024-08-17T02:36:55.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-08-17T02:36:55.521 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-08-17T02:36:55.525 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-08-17T02:36:55.615 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-08-17T02:36:55.618 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-17T02:36:55.618 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-08-17T02:36:55.660 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-08-17T02:36:55.660 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-08-17T02:36:55.702 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-08-17T02:36:55.702 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive'} 2024-08-17T02:36:55.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-08-17T02:36:55.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:36:55.761 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-08-17T02:36:55.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:36:55.819 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-17T02:36:55.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-08-17T02:36:55.877 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-08-17T02:36:55.943 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-17T02:36:55.952 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-17T02:36:55.963 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-17T02:36:55.964 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi142.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-17T02:42:34.333 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com')] 2024-08-17T02:42:34.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-08-17T02:42:34.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:42:34.434 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-08-17T02:42:34.532 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-08-17T02:42:34.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-08-17T02:42:34.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-17T02:42:34.631 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-08-17T02:42:34.758 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-08-17T02:42:34.759 INFO:teuthology.run_tasks:Running task clock... 2024-08-17T02:42:34.769 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-17T02:42:34.769 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-17T02:42:34.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-17T02:42:34.774 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-17T02:42:34.774 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-17T02:42:34.834 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-17T02:42:34.863 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-17T02:42:34.864 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-17T02:42:34.894 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-17T02:42:34.915 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-08-17T02:42:34.939 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-08-17T02:42:34.944 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-08-17T02:42:34.968 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-08-17T02:42:34.969 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-17T02:42:34.996 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-17T02:42:34.998 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-17T02:42:35.025 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-17T02:42:35.095 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-08-17T02:42:35.102 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-17T02:42:35.102 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-08-17T02:42:35.102 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-17T02:42:35.102 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-17T02:42:35.102 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-17T02:42:35.102 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-17T02:42:35.116 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-08-17T02:42:35.122 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-17T02:42:35.123 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-08-17T02:42:35.123 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-17T02:42:35.123 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-17T02:42:35.123 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-17T02:42:35.123 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-17T02:42:35.124 INFO:teuthology.run_tasks:Running task install... 2024-08-17T02:42:35.133 DEBUG:teuthology.task.install:project ceph 2024-08-17T02:42:35.133 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e'}} 2024-08-17T02:42:35.133 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e'} 2024-08-17T02:42:35.133 INFO:teuthology.task.install:Using flavor: default 2024-08-17T02:42:35.139 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-17T02:42:35.139 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-08-17T02:42:35.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo dnf -y install 'dnf-command(copr)' 2024-08-17T02:42:35.140 DEBUG:teuthology.orchestra.run.smithi142:> sudo dnf -y install 'dnf-command(copr)' 2024-08-17T02:42:35.836 INFO:teuthology.orchestra.run.smithi142.stdout:Last metadata expiration check: 0:01:20 ago on Sat 17 Aug 2024 02:41:15 AM UTC. 2024-08-17T02:42:35.867 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:19 ago on Sat 17 Aug 2024 02:41:16 AM UTC. 2024-08-17T02:42:35.924 INFO:teuthology.orchestra.run.smithi142.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-08-17T02:42:35.959 INFO:teuthology.orchestra.run.smithi016.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-08-17T02:42:35.967 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-08-17T02:42:35.968 INFO:teuthology.orchestra.run.smithi142.stdout:Nothing to do. 2024-08-17T02:42:35.968 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-08-17T02:42:36.003 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-08-17T02:42:36.004 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2024-08-17T02:42:36.005 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-08-17T02:42:36.036 DEBUG:teuthology.orchestra.run.smithi142:> sudo dnf -y copr enable ceph/el9 2024-08-17T02:42:36.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo dnf -y copr enable ceph/el9 2024-08-17T02:42:36.294 INFO:teuthology.orchestra.run.smithi142.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-08-17T02:42:36.294 INFO:teuthology.orchestra.run.smithi142.stderr:of the main distribution, and quality may vary. 2024-08-17T02:42:36.294 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-08-17T02:42:36.294 INFO:teuthology.orchestra.run.smithi142.stderr:The Fedora Project does not exercise any power over the contents of 2024-08-17T02:42:36.294 INFO:teuthology.orchestra.run.smithi142.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-08-17T02:42:36.294 INFO:teuthology.orchestra.run.smithi142.stderr:, 2024-08-17T02:42:36.294 INFO:teuthology.orchestra.run.smithi142.stderr:and packages are not held to any quality or security level. 2024-08-17T02:42:36.295 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-08-17T02:42:36.295 INFO:teuthology.orchestra.run.smithi142.stderr:Please do not file bug reports about these packages in Fedora 2024-08-17T02:42:36.295 INFO:teuthology.orchestra.run.smithi142.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-08-17T02:42:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-08-17T02:42:36.341 INFO:teuthology.orchestra.run.smithi016.stderr:of the main distribution, and quality may vary. 2024-08-17T02:42:36.342 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-08-17T02:42:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:The Fedora Project does not exercise any power over the contents of 2024-08-17T02:42:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-08-17T02:42:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:, 2024-08-17T02:42:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:and packages are not held to any quality or security level. 2024-08-17T02:42:36.342 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-08-17T02:42:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:Please do not file bug reports about these packages in Fedora 2024-08-17T02:42:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-08-17T02:42:36.375 INFO:teuthology.orchestra.run.smithi142.stdout:Repository successfully enabled. 2024-08-17T02:42:36.421 INFO:teuthology.orchestra.run.smithi016.stdout:Repository successfully enabled. 2024-08-17T02:42:36.422 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e', 'tag': None, 'wait_for_package': False} 2024-08-17T02:42:36.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2577e8139c6b8786f9d6d6ea21b8da9244b9260e 2024-08-17T02:42:36.466 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e', 'tag': None, 'wait_for_package': False} 2024-08-17T02:42:36.467 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2577e8139c6b8786f9d6d6ea21b8da9244b9260e 2024-08-17T02:42:36.576 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/centos/9/flavors/default/ 2024-08-17T02:42:36.576 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1375.g2577e813 2024-08-17T02:42:36.615 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/centos/9/flavors/default/ 2024-08-17T02:42:36.615 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1375.g2577e813 2024-08-17T02:42:36.685 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-17T02:42:36.685 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:42:36.685 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-17T02:42:36.725 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-mchangir-mgr-volumes-print-thread-name-quincy/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-17T02:42:36.725 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:42:36.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-17T02:42:36.740 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2024-08-17T02:42:36.741 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-17T02:42:36.783 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2024-08-17T02:42:36.783 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2577e8139c6b8786f9d6d6ea21b8da9244b9260e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-17T02:42:36.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-17T02:42:36.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-17T02:42:36.915 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-17T02:42:36.962 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-17T02:42:36.970 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-08-17T02:42:36.973 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-08-17T02:42:37.017 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-08-17T02:42:37.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-08-17T02:42:37.249 INFO:teuthology.orchestra.run.smithi142.stdout:45 files removed 2024-08-17T02:42:37.282 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2024-08-17T02:42:37.335 INFO:teuthology.orchestra.run.smithi016.stdout:45 files removed 2024-08-17T02:42:37.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2024-08-17T02:42:37.978 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 232 kB/s | 77 kB 00:00 2024-08-17T02:42:38.069 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 238 kB/s | 77 kB 00:00 2024-08-17T02:42:38.329 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-08-17T02:42:38.436 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 33 kB/s | 11 kB 00:00 2024-08-17T02:42:38.671 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 5.9 kB/s | 1.8 kB 00:00 2024-08-17T02:42:38.778 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 5.9 kB/s | 1.8 kB 00:00 2024-08-17T02:42:38.972 INFO:teuthology.orchestra.run.smithi142.stdout:Copr repo for el9 owned by ceph 29 kB/s | 7.6 kB 00:00 2024-08-17T02:42:39.012 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for el9 owned by ceph 39 kB/s | 7.6 kB 00:00 2024-08-17T02:42:39.538 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-08-17T02:42:40.087 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.2 MB 00:01 2024-08-17T02:42:42.915 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 20 MB 00:02 2024-08-17T02:42:43.047 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-08-17T02:42:47.549 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.6 MB 00:00 2024-08-17T02:42:48.897 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.6 MB 00:01 2024-08-17T02:42:49.024 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 113 kB/s | 18 kB 00:00 2024-08-17T02:42:50.078 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 24 MB/s | 22 MB 00:00 2024-08-17T02:42:50.673 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 18 kB 00:00 2024-08-17T02:42:51.686 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 26 MB/s | 22 MB 00:00 2024-08-17T02:42:56.326 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-08-17T02:42:58.242 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-08-17T02:42:58.916 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-17T02:42:58.917 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-17T02:42:58.927 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-08-17T02:42:58.928 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-17T02:42:58.929 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-17T02:42:58.930 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-17T02:42:58.931 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-17T02:42:58.933 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-17T02:42:58.935 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-08-17T02:42:58.994 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-08-17T02:42:59.013 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-08-17T02:42:59.013 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-08-17T02:42:59.013 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-08-17T02:42:59.013 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-08-17T02:42:59.013 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-17T02:42:59.013 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:17.2.7-1375.g2577e813.el9 ceph 6.5 k 2024-08-17T02:42:59.013 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:17.2.7-1375.g2577e813.el9 ceph 5.0 M 2024-08-17T02:42:59.013 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:17.2.7-1375.g2577e813.el9 ceph 813 k 2024-08-17T02:42:59.014 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1375.g2577e813.el9 ceph 137 k 2024-08-17T02:42:59.014 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:17.2.7-1375.g2577e813.el9 ceph 1.4 M 2024-08-17T02:42:59.014 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 124 k 2024-08-17T02:42:59.014 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 3.4 M 2024-08-17T02:42:59.014 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 7.4 M 2024-08-17T02:42:59.014 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 47 k 2024-08-17T02:42:59.014 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:17.2.7-1375.g2577e813.el9 ceph 14 M 2024-08-17T02:42:59.014 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:17.2.7-1375.g2577e813.el9 ceph 48 M 2024-08-17T02:42:59.014 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 269 k 2024-08-17T02:42:59.014 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 83 k 2024-08-17T02:42:59.014 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:17.2.7-1375.g2577e813.el9 ceph 215 k 2024-08-17T02:42:59.014 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 18 k 2024-08-17T02:42:59.015 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-17T02:42:59.015 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-17T02:42:59.015 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-17T02:42:59.015 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:17.2.7-1375.g2577e813.el9 ceph 25 k 2024-08-17T02:42:59.015 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:17.2.7-1375.g2577e813.el9 ceph 636 k 2024-08-17T02:42:59.015 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:17.2.7-1375.g2577e813.el9 ceph 126 k 2024-08-17T02:42:59.015 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-17T02:42:59.015 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-17T02:42:59.015 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-17T02:42:59.015 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:17.2.7-1375.g2577e813.el9 ceph 182 k 2024-08-17T02:42:59.015 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:17.2.7-1375.g2577e813.el9 ceph 369 k 2024-08-17T02:42:59.016 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:17.2.7-1375.g2577e813.el9 ceph 344 k 2024-08-17T02:42:59.016 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:17.2.7-1375.g2577e813.el9 ceph 116 k 2024-08-17T02:42:59.016 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:17.2.7-1375.g2577e813.el9 ceph 82 k 2024-08-17T02:42:59.016 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:17.2.7-1375.g2577e813.el9 ceph 2.9 M 2024-08-17T02:42:59.016 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:17.2.7-1375.g2577e813.el9 ceph 165 k 2024-08-17T02:42:59.016 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-17T02:42:59.016 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-08-17T02:42:59.016 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-08-17T02:42:59.016 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-08-17T02:42:59.016 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-08-17T02:42:59.016 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-08-17T02:42:59.016 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:17.2.7-1375.g2577e813.el9 ceph 3.1 M 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:17.2.7-1375.g2577e813.el9 ceph 2.9 M 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:17.2.7-1375.g2577e813.el9 ceph 22 M 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 24 k 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:17.2.7-1375.g2577e813.el9 ceph 2.1 M 2024-08-17T02:42:59.017 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 240 k 2024-08-17T02:42:59.018 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:17.2.7-1375.g2577e813.el9 ceph 3.9 M 2024-08-17T02:42:59.018 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:17.2.7-1375.g2577e813.el9 ceph 15 M 2024-08-17T02:42:59.018 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 15 k 2024-08-17T02:42:59.018 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:17.2.7-1375.g2577e813.el9 ceph 25 k 2024-08-17T02:42:59.018 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-17T02:42:59.018 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-17T02:42:59.018 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-17T02:42:59.018 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-17T02:42:59.018 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-17T02:42:59.018 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:17.2.7-1375.g2577e813.el9 ceph 163 k 2024-08-17T02:42:59.018 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-17T02:42:59.018 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-17T02:42:59.022 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-17T02:42:59.022 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-17T02:42:59.022 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-17T02:42:59.022 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:17.2.7-1375.g2577e813.el9 ceph 458 k 2024-08-17T02:42:59.022 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-17T02:42:59.022 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:17.2.7-1375.g2577e813.el9 ceph 4.9 M 2024-08-17T02:42:59.022 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-17T02:42:59.022 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-17T02:42:59.022 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:17.2.7-1375.g2577e813.el9 ceph 45 k 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:17.2.7-1375.g2577e813.el9 ceph 120 k 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-17T02:42:59.023 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-17T02:42:59.024 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-17T02:42:59.025 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-17T02:42:59.026 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-17T02:42:59.026 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-17T02:42:59.026 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-17T02:42:59.026 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-17T02:42:59.026 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-17T02:42:59.026 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-17T02:42:59.026 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-17T02:42:59.026 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-17T02:42:59.026 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-17T02:42:59.026 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-17T02:42:59.026 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-17T02:42:59.026 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:42:59.027 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-08-17T02:42:59.028 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-08-17T02:42:59.028 INFO:teuthology.orchestra.run.smithi142.stdout:Install 121 Packages 2024-08-17T02:42:59.028 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 12 Packages 2024-08-17T02:42:59.028 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:42:59.028 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 206 M 2024-08-17T02:42:59.028 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-08-17T02:42:59.593 INFO:teuthology.orchestra.run.smithi142.stdout:(1/133): ceph-17.2.7-1375.g2577e813.el9.x86_64. 22 kB/s | 6.5 kB 00:00 2024-08-17T02:42:59.954 INFO:teuthology.orchestra.run.smithi142.stdout:(2/133): ceph-fuse-17.2.7-1375.g2577e813.el9.x8 2.2 MB/s | 813 kB 00:00 2024-08-17T02:43:00.036 INFO:teuthology.orchestra.run.smithi142.stdout:(3/133): ceph-immutable-object-cache-17.2.7-137 1.7 MB/s | 137 kB 00:00 2024-08-17T02:43:00.344 INFO:teuthology.orchestra.run.smithi142.stdout:(4/133): ceph-base-17.2.7-1375.g2577e813.el9.x8 4.8 MB/s | 5.0 MB 00:01 2024-08-17T02:43:00.495 INFO:teuthology.orchestra.run.smithi142.stdout:(5/133): ceph-mgr-17.2.7-1375.g2577e813.el9.x86 9.4 MB/s | 1.4 MB 00:00 2024-08-17T02:43:00.536 INFO:teuthology.orchestra.run.smithi142.stdout:(6/133): ceph-mds-17.2.7-1375.g2577e813.el9.x86 4.1 MB/s | 2.1 MB 00:00 2024-08-17T02:43:00.837 INFO:teuthology.orchestra.run.smithi142.stdout:(7/133): ceph-mon-17.2.7-1375.g2577e813.el9.x86 11 MB/s | 3.9 MB 00:00 2024-08-17T02:43:00.888 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-17T02:43:00.889 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-17T02:43:00.900 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-08-17T02:43:00.901 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-17T02:43:00.902 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-17T02:43:00.902 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-17T02:43:00.904 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-17T02:43:00.907 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-17T02:43:00.908 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-08-17T02:43:00.973 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-08-17T02:43:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-08-17T02:43:00.992 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-08-17T02:43:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-08-17T02:43:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-08-17T02:43:00.992 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-17T02:43:00.992 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:17.2.7-1375.g2577e813.el9 ceph 6.5 k 2024-08-17T02:43:00.992 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:17.2.7-1375.g2577e813.el9 ceph 5.0 M 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:17.2.7-1375.g2577e813.el9 ceph 813 k 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1375.g2577e813.el9 ceph 137 k 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:17.2.7-1375.g2577e813.el9 ceph 1.4 M 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 124 k 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 3.4 M 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 7.4 M 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 47 k 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:17.2.7-1375.g2577e813.el9 ceph 14 M 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:17.2.7-1375.g2577e813.el9 ceph 48 M 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 269 k 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 83 k 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:17.2.7-1375.g2577e813.el9 ceph 215 k 2024-08-17T02:43:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 18 k 2024-08-17T02:43:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-17T02:43:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-17T02:43:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-17T02:43:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:17.2.7-1375.g2577e813.el9 ceph 25 k 2024-08-17T02:43:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:17.2.7-1375.g2577e813.el9 ceph 636 k 2024-08-17T02:43:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:17.2.7-1375.g2577e813.el9 ceph 126 k 2024-08-17T02:43:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-17T02:43:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-17T02:43:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-17T02:43:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:17.2.7-1375.g2577e813.el9 ceph 182 k 2024-08-17T02:43:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:17.2.7-1375.g2577e813.el9 ceph 369 k 2024-08-17T02:43:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:17.2.7-1375.g2577e813.el9 ceph 344 k 2024-08-17T02:43:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:17.2.7-1375.g2577e813.el9 ceph 116 k 2024-08-17T02:43:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:17.2.7-1375.g2577e813.el9 ceph 82 k 2024-08-17T02:43:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:17.2.7-1375.g2577e813.el9 ceph 2.9 M 2024-08-17T02:43:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:17.2.7-1375.g2577e813.el9 ceph 165 k 2024-08-17T02:43:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-17T02:43:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-08-17T02:43:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-08-17T02:43:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-08-17T02:43:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-08-17T02:43:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:17.2.7-1375.g2577e813.el9 ceph 3.1 M 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:17.2.7-1375.g2577e813.el9 ceph 2.9 M 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:17.2.7-1375.g2577e813.el9 ceph 22 M 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 24 k 2024-08-17T02:43:00.996 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:17.2.7-1375.g2577e813.el9 ceph 2.1 M 2024-08-17T02:43:00.997 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 240 k 2024-08-17T02:43:00.997 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:17.2.7-1375.g2577e813.el9 ceph 3.9 M 2024-08-17T02:43:00.998 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:17.2.7-1375.g2577e813.el9 ceph 15 M 2024-08-17T02:43:00.998 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1375.g2577e813.el9 ceph-noarch 15 k 2024-08-17T02:43:00.998 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:17.2.7-1375.g2577e813.el9 ceph 25 k 2024-08-17T02:43:00.998 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-17T02:43:00.998 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-17T02:43:00.998 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-17T02:43:00.998 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:17.2.7-1375.g2577e813.el9 ceph 163 k 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:17.2.7-1375.g2577e813.el9 ceph 458 k 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:17.2.7-1375.g2577e813.el9 ceph 4.9 M 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-17T02:43:00.999 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-17T02:43:01.000 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-17T02:43:01.000 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-17T02:43:01.000 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-17T02:43:01.000 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-17T02:43:01.000 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-17T02:43:01.000 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-17T02:43:01.000 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:17.2.7-1375.g2577e813.el9 ceph 45 k 2024-08-17T02:43:01.000 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:17.2.7-1375.g2577e813.el9 ceph 120 k 2024-08-17T02:43:01.000 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-17T02:43:01.000 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-17T02:43:01.000 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-17T02:43:01.000 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-08-17T02:43:01.001 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-17T02:43:01.002 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-17T02:43:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-08-17T02:43:01.004 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:43:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-08-17T02:43:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-08-17T02:43:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:Install 121 Packages 2024-08-17T02:43:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 12 Packages 2024-08-17T02:43:01.005 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:43:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 206 M 2024-08-17T02:43:01.005 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-08-17T02:43:01.602 INFO:teuthology.orchestra.run.smithi016.stdout:(1/133): ceph-17.2.7-1375.g2577e813.el9.x86_64. 23 kB/s | 6.5 kB 00:00 2024-08-17T02:43:01.685 INFO:teuthology.orchestra.run.smithi142.stdout:(8/133): ceph-common-17.2.7-1375.g2577e813.el9. 9.4 MB/s | 22 MB 00:02 2024-08-17T02:43:01.802 INFO:teuthology.orchestra.run.smithi016.stdout:(2/133): ceph-fuse-17.2.7-1375.g2577e813.el9.x8 4.0 MB/s | 813 kB 00:00 2024-08-17T02:43:01.871 INFO:teuthology.orchestra.run.smithi142.stdout:(9/133): ceph-radosgw-17.2.7-1375.g2577e813.el9 14 MB/s | 14 MB 00:01 2024-08-17T02:43:01.877 INFO:teuthology.orchestra.run.smithi016.stdout:(3/133): ceph-immutable-object-cache-17.2.7-137 1.8 MB/s | 137 kB 00:00 2024-08-17T02:43:01.887 INFO:teuthology.orchestra.run.smithi142.stdout:(10/133): ceph-selinux-17.2.7-1375.g2577e813.el 125 kB/s | 25 kB 00:00 2024-08-17T02:43:01.996 INFO:teuthology.orchestra.run.smithi142.stdout:(11/133): cephfs-mirror-17.2.7-1375.g2577e813.e 1.9 MB/s | 215 kB 00:00 2024-08-17T02:43:02.061 INFO:teuthology.orchestra.run.smithi016.stdout:(4/133): ceph-base-17.2.7-1375.g2577e813.el9.x8 6.6 MB/s | 5.0 MB 00:00 2024-08-17T02:43:02.097 INFO:teuthology.orchestra.run.smithi142.stdout:(12/133): libcephfs-devel-17.2.7-1375.g2577e813 250 kB/s | 25 kB 00:00 2024-08-17T02:43:02.145 INFO:teuthology.orchestra.run.smithi016.stdout:(5/133): ceph-mds-17.2.7-1375.g2577e813.el9.x86 7.8 MB/s | 2.1 MB 00:00 2024-08-17T02:43:02.203 INFO:teuthology.orchestra.run.smithi016.stdout:(6/133): ceph-mgr-17.2.7-1375.g2577e813.el9.x86 9.9 MB/s | 1.4 MB 00:00 2024-08-17T02:43:02.263 INFO:teuthology.orchestra.run.smithi142.stdout:(13/133): ceph-osd-17.2.7-1375.g2577e813.el9.x8 8.9 MB/s | 15 MB 00:01 2024-08-17T02:43:02.297 INFO:teuthology.orchestra.run.smithi142.stdout:(14/133): libcephfs2-17.2.7-1375.g2577e813.el9. 3.1 MB/s | 636 kB 00:00 2024-08-17T02:43:02.365 INFO:teuthology.orchestra.run.smithi142.stdout:(15/133): libcephsqlite-17.2.7-1375.g2577e813.e 1.6 MB/s | 163 kB 00:00 2024-08-17T02:43:02.397 INFO:teuthology.orchestra.run.smithi142.stdout:(16/133): librados-devel-17.2.7-1375.g2577e813. 1.2 MB/s | 126 kB 00:00 2024-08-17T02:43:02.448 INFO:teuthology.orchestra.run.smithi142.stdout:(17/133): libradosstriper1-17.2.7-1375.g2577e81 5.5 MB/s | 458 kB 00:00 2024-08-17T02:43:02.470 INFO:teuthology.orchestra.run.smithi016.stdout:(7/133): ceph-mon-17.2.7-1375.g2577e813.el9.x86 12 MB/s | 3.9 MB 00:00 2024-08-17T02:43:02.531 INFO:teuthology.orchestra.run.smithi142.stdout:(18/133): python3-ceph-argparse-17.2.7-1375.g25 541 kB/s | 45 kB 00:00 2024-08-17T02:43:02.890 INFO:teuthology.orchestra.run.smithi142.stdout:(19/133): librgw2-17.2.7-1375.g2577e813.el9.x86 10 MB/s | 4.9 MB 00:00 2024-08-17T02:43:02.974 INFO:teuthology.orchestra.run.smithi142.stdout:(20/133): python3-cephfs-17.2.7-1375.g2577e813. 2.1 MB/s | 182 kB 00:00 2024-08-17T02:43:03.082 INFO:teuthology.orchestra.run.smithi142.stdout:(21/133): python3-rados-17.2.7-1375.g2577e813.e 3.3 MB/s | 369 kB 00:00 2024-08-17T02:43:03.116 INFO:teuthology.orchestra.run.smithi142.stdout:(22/133): python3-ceph-common-17.2.7-1375.g2577 206 kB/s | 120 kB 00:00 2024-08-17T02:43:03.141 INFO:teuthology.orchestra.run.smithi142.stdout:(23/133): python3-rbd-17.2.7-1375.g2577e813.el9 5.8 MB/s | 344 kB 00:00 2024-08-17T02:43:03.166 INFO:teuthology.orchestra.run.smithi142.stdout:(24/133): python3-rgw-17.2.7-1375.g2577e813.el9 2.3 MB/s | 116 kB 00:00 2024-08-17T02:43:03.191 INFO:teuthology.orchestra.run.smithi142.stdout:(25/133): rbd-fuse-17.2.7-1375.g2577e813.el9.x8 1.6 MB/s | 82 kB 00:00 2024-08-17T02:43:03.204 INFO:teuthology.orchestra.run.smithi016.stdout:(8/133): ceph-osd-17.2.7-1375.g2577e813.el9.x86 15 MB/s | 15 MB 00:00 2024-08-17T02:43:03.250 INFO:teuthology.orchestra.run.smithi142.stdout:(26/133): rbd-nbd-17.2.7-1375.g2577e813.el9.x86 2.8 MB/s | 165 kB 00:00 2024-08-17T02:43:03.279 INFO:teuthology.orchestra.run.smithi016.stdout:(9/133): ceph-selinux-17.2.7-1375.g2577e813.el9 338 kB/s | 25 kB 00:00 2024-08-17T02:43:03.325 INFO:teuthology.orchestra.run.smithi142.stdout:(27/133): ceph-grafana-dashboards-17.2.7-1375.g 326 kB/s | 24 kB 00:00 2024-08-17T02:43:03.451 INFO:teuthology.orchestra.run.smithi142.stdout:(28/133): rbd-mirror-17.2.7-1375.g2577e813.el9. 10 MB/s | 2.9 MB 00:00 2024-08-17T02:43:03.476 INFO:teuthology.orchestra.run.smithi142.stdout:(29/133): ceph-mgr-cephadm-17.2.7-1375.g2577e81 825 kB/s | 124 kB 00:00 2024-08-17T02:43:03.737 INFO:teuthology.orchestra.run.smithi016.stdout:(10/133): ceph-radosgw-17.2.7-1375.g2577e813.el 11 MB/s | 14 MB 00:01 2024-08-17T02:43:03.826 INFO:teuthology.orchestra.run.smithi142.stdout:(30/133): ceph-mgr-dashboard-17.2.7-1375.g2577e 8.9 MB/s | 3.4 MB 00:00 2024-08-17T02:43:03.829 INFO:teuthology.orchestra.run.smithi016.stdout:(11/133): cephfs-mirror-17.2.7-1375.g2577e813.e 2.3 MB/s | 215 kB 00:00 2024-08-17T02:43:03.904 INFO:teuthology.orchestra.run.smithi016.stdout:(12/133): libcephfs-devel-17.2.7-1375.g2577e813 334 kB/s | 25 kB 00:00 2024-08-17T02:43:03.918 INFO:teuthology.orchestra.run.smithi142.stdout:(31/133): ceph-mgr-modules-core-17.2.7-1375.g25 2.6 MB/s | 240 kB 00:00 2024-08-17T02:43:04.005 INFO:teuthology.orchestra.run.smithi016.stdout:(13/133): libcephfs2-17.2.7-1375.g2577e813.el9. 6.2 MB/s | 636 kB 00:00 2024-08-17T02:43:04.019 INFO:teuthology.orchestra.run.smithi142.stdout:(32/133): ceph-mgr-rook-17.2.7-1375.g2577e813.e 475 kB/s | 47 kB 00:00 2024-08-17T02:43:04.080 INFO:teuthology.orchestra.run.smithi016.stdout:(14/133): libcephsqlite-17.2.7-1375.g2577e813.e 2.1 MB/s | 163 kB 00:00 2024-08-17T02:43:04.110 INFO:teuthology.orchestra.run.smithi142.stdout:(33/133): ceph-prometheus-alerts-17.2.7-1375.g2 160 kB/s | 15 kB 00:00 2024-08-17T02:43:04.164 INFO:teuthology.orchestra.run.smithi016.stdout:(15/133): librados-devel-17.2.7-1375.g2577e813. 1.5 MB/s | 126 kB 00:00 2024-08-17T02:43:04.202 INFO:teuthology.orchestra.run.smithi142.stdout:(34/133): ceph-mgr-diskprediction-local-17.2.7- 10 MB/s | 7.4 MB 00:00 2024-08-17T02:43:04.227 INFO:teuthology.orchestra.run.smithi142.stdout:(35/133): ceph-volume-17.2.7-1375.g2577e813.el9 2.3 MB/s | 269 kB 00:00 2024-08-17T02:43:04.239 INFO:teuthology.orchestra.run.smithi016.stdout:(16/133): libradosstriper1-17.2.7-1375.g2577e81 6.0 MB/s | 458 kB 00:00 2024-08-17T02:43:04.278 INFO:teuthology.orchestra.run.smithi142.stdout:(36/133): cephadm-17.2.7-1375.g2577e813.el9.noa 1.1 MB/s | 83 kB 00:00 2024-08-17T02:43:04.303 INFO:teuthology.orchestra.run.smithi142.stdout:(37/133): cephfs-top-17.2.7-1375.g2577e813.el9. 241 kB/s | 18 kB 00:00 2024-08-17T02:43:04.437 INFO:teuthology.orchestra.run.smithi142.stdout:(38/133): fuse-2.9.9-16.el9.x86_64.rpm 503 kB/s | 80 kB 00:00 2024-08-17T02:43:04.470 INFO:teuthology.orchestra.run.smithi142.stdout:(39/133): libgfortran-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 803 kB 00:00 2024-08-17T02:43:04.504 INFO:teuthology.orchestra.run.smithi142.stdout:(40/133): libquadmath-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 195 kB 00:00 2024-08-17T02:43:04.529 INFO:teuthology.orchestra.run.smithi142.stdout:(41/133): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-08-17T02:43:04.588 INFO:teuthology.orchestra.run.smithi142.stdout:(42/133): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-08-17T02:43:04.613 INFO:teuthology.orchestra.run.smithi142.stdout:(43/133): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-08-17T02:43:04.647 INFO:teuthology.orchestra.run.smithi142.stdout:(44/133): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-08-17T02:43:04.647 INFO:teuthology.orchestra.run.smithi016.stdout:(17/133): librgw2-17.2.7-1375.g2577e813.el9.x86 12 MB/s | 4.9 MB 00:00 2024-08-17T02:43:04.674 INFO:teuthology.orchestra.run.smithi142.stdout:(45/133): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-08-17T02:43:04.697 INFO:teuthology.orchestra.run.smithi142.stdout:(46/133): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-08-17T02:43:04.715 INFO:teuthology.orchestra.run.smithi016.stdout:(18/133): python3-ceph-argparse-17.2.7-1375.g25 678 kB/s | 45 kB 00:00 2024-08-17T02:43:04.790 INFO:teuthology.orchestra.run.smithi016.stdout:(19/133): python3-ceph-common-17.2.7-1375.g2577 1.6 MB/s | 120 kB 00:00 2024-08-17T02:43:04.865 INFO:teuthology.orchestra.run.smithi016.stdout:(20/133): python3-cephfs-17.2.7-1375.g2577e813. 2.4 MB/s | 182 kB 00:00 2024-08-17T02:43:04.933 INFO:teuthology.orchestra.run.smithi142.stdout:(47/133): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-08-17T02:43:04.982 INFO:teuthology.orchestra.run.smithi142.stdout:(48/133): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-08-17T02:43:04.998 INFO:teuthology.orchestra.run.smithi142.stdout:(49/133): elfutils-libelf-devel-0.191-4.el9.x86 324 kB/s | 22 kB 00:00 2024-08-17T02:43:05.048 INFO:teuthology.orchestra.run.smithi142.stdout:(50/133): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-08-17T02:43:05.065 INFO:teuthology.orchestra.run.smithi142.stdout:(51/133): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-08-17T02:43:05.124 INFO:teuthology.orchestra.run.smithi016.stdout:(21/133): ceph-common-17.2.7-1375.g2577e813.el9 5.9 MB/s | 22 MB 00:03 2024-08-17T02:43:05.124 INFO:teuthology.orchestra.run.smithi142.stdout:(52/133): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-08-17T02:43:05.174 INFO:teuthology.orchestra.run.smithi016.stdout:(22/133): python3-rados-17.2.7-1375.g2577e813.e 1.2 MB/s | 369 kB 00:00 2024-08-17T02:43:05.175 INFO:teuthology.orchestra.run.smithi142.stdout:(53/133): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-08-17T02:43:05.208 INFO:teuthology.orchestra.run.smithi016.stdout:(23/133): python3-rbd-17.2.7-1375.g2577e813.el9 4.0 MB/s | 344 kB 00:00 2024-08-17T02:43:05.233 INFO:teuthology.orchestra.run.smithi016.stdout:(24/133): python3-rgw-17.2.7-1375.g2577e813.el9 1.9 MB/s | 116 kB 00:00 2024-08-17T02:43:05.233 INFO:teuthology.orchestra.run.smithi142.stdout:(54/133): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-08-17T02:43:05.258 INFO:teuthology.orchestra.run.smithi016.stdout:(25/133): rbd-fuse-17.2.7-1375.g2577e813.el9.x8 1.6 MB/s | 82 kB 00:00 2024-08-17T02:43:05.283 INFO:teuthology.orchestra.run.smithi142.stdout:(55/133): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-08-17T02:43:05.300 INFO:teuthology.orchestra.run.smithi142.stdout:(56/133): librabbitmq-0.11.0-7.el9.x86_64.rpm 678 kB/s | 45 kB 00:00 2024-08-17T02:43:05.309 INFO:teuthology.orchestra.run.smithi016.stdout:(26/133): rbd-nbd-17.2.7-1375.g2577e813.el9.x86 3.2 MB/s | 165 kB 00:00 2024-08-17T02:43:05.351 INFO:teuthology.orchestra.run.smithi142.stdout:(57/133): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-08-17T02:43:05.384 INFO:teuthology.orchestra.run.smithi142.stdout:(58/133): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-08-17T02:43:05.394 INFO:teuthology.orchestra.run.smithi016.stdout:(27/133): ceph-grafana-dashboards-17.2.7-1375.g 295 kB/s | 24 kB 00:00 2024-08-17T02:43:05.401 INFO:teuthology.orchestra.run.smithi142.stdout:(59/133): libuuid-devel-2.37.4-18.el9.x86_64.rp 511 kB/s | 25 kB 00:00 2024-08-17T02:43:05.460 INFO:teuthology.orchestra.run.smithi142.stdout:(60/133): libzstd-devel-1.5.1-2.el9.x86_64.rpm 633 kB/s | 47 kB 00:00 2024-08-17T02:43:05.485 INFO:teuthology.orchestra.run.smithi142.stdout:(61/133): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-08-17T02:43:05.492 INFO:teuthology.orchestra.run.smithi016.stdout:(28/133): rbd-mirror-17.2.7-1375.g2577e813.el9. 11 MB/s | 2.9 MB 00:00 2024-08-17T02:43:05.511 INFO:teuthology.orchestra.run.smithi142.stdout:(62/133): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-08-17T02:43:05.526 INFO:teuthology.orchestra.run.smithi016.stdout:(29/133): ceph-mgr-cephadm-17.2.7-1375.g2577e81 927 kB/s | 124 kB 00:00 2024-08-17T02:43:05.844 INFO:teuthology.orchestra.run.smithi142.stdout:(63/133): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-08-17T02:43:05.919 INFO:teuthology.orchestra.run.smithi142.stdout:(64/133): openssl-devel-3.2.2-2.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-08-17T02:43:05.986 INFO:teuthology.orchestra.run.smithi142.stdout:(65/133): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-08-17T02:43:06.053 INFO:teuthology.orchestra.run.smithi142.stdout:(66/133): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-08-17T02:43:06.119 INFO:teuthology.orchestra.run.smithi016.stdout:(30/133): ceph-test-17.2.7-1375.g2577e813.el9.x 17 MB/s | 48 MB 00:02 2024-08-17T02:43:06.177 INFO:teuthology.orchestra.run.smithi016.stdout:(31/133): ceph-mgr-dashboard-17.2.7-1375.g2577e 4.9 MB/s | 3.4 MB 00:00 2024-08-17T02:43:06.202 INFO:teuthology.orchestra.run.smithi016.stdout:(32/133): ceph-mgr-modules-core-17.2.7-1375.g25 2.8 MB/s | 240 kB 00:00 2024-08-17T02:43:06.228 INFO:teuthology.orchestra.run.smithi016.stdout:(33/133): ceph-mgr-rook-17.2.7-1375.g2577e813.e 952 kB/s | 47 kB 00:00 2024-08-17T02:43:06.253 INFO:teuthology.orchestra.run.smithi016.stdout:(34/133): ceph-prometheus-alerts-17.2.7-1375.g2 292 kB/s | 15 kB 00:00 2024-08-17T02:43:06.278 INFO:teuthology.orchestra.run.smithi016.stdout:(35/133): ceph-volume-17.2.7-1375.g2577e813.el9 5.2 MB/s | 269 kB 00:00 2024-08-17T02:43:06.303 INFO:teuthology.orchestra.run.smithi016.stdout:(36/133): cephadm-17.2.7-1375.g2577e813.el9.noa 1.6 MB/s | 83 kB 00:00 2024-08-17T02:43:06.329 INFO:teuthology.orchestra.run.smithi016.stdout:(37/133): cephfs-top-17.2.7-1375.g2577e813.el9. 363 kB/s | 18 kB 00:00 2024-08-17T02:43:06.520 INFO:teuthology.orchestra.run.smithi142.stdout:(67/133): ceph-test-17.2.7-1375.g2577e813.el9.x 10 MB/s | 48 MB 00:04 2024-08-17T02:43:06.545 INFO:teuthology.orchestra.run.smithi142.stdout:(68/133): python3-jsonpatch-1.21-16.el9.noarch. 52 kB/s | 26 kB 00:00 2024-08-17T02:43:06.562 INFO:teuthology.orchestra.run.smithi016.stdout:(38/133): ceph-mgr-diskprediction-local-17.2.7- 7.1 MB/s | 7.4 MB 00:01 2024-08-17T02:43:06.638 INFO:teuthology.orchestra.run.smithi142.stdout:(69/133): python3-babel-2.9.1-2.el9.noarch.rpm 7.5 MB/s | 6.0 MB 00:00 2024-08-17T02:43:06.663 INFO:teuthology.orchestra.run.smithi142.stdout:(70/133): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-08-17T02:43:06.688 INFO:teuthology.orchestra.run.smithi142.stdout:(71/133): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-08-17T02:43:06.713 INFO:teuthology.orchestra.run.smithi142.stdout:(72/133): python3-jsonpointer-2.0-4.el9.noarch. 100 kB/s | 19 kB 00:00 2024-08-17T02:43:06.721 INFO:teuthology.orchestra.run.smithi016.stdout:(39/133): fuse-2.9.9-16.el9.x86_64.rpm 191 kB/s | 80 kB 00:00 2024-08-17T02:43:06.747 INFO:teuthology.orchestra.run.smithi142.stdout:(73/133): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-08-17T02:43:06.805 INFO:teuthology.orchestra.run.smithi142.stdout:(74/133): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-08-17T02:43:06.864 INFO:teuthology.orchestra.run.smithi142.stdout:(75/133): python3-prettytable-0.7.2-27.el9.noar 726 kB/s | 42 kB 00:00 2024-08-17T02:43:06.889 INFO:teuthology.orchestra.run.smithi142.stdout:(76/133): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-08-17T02:43:06.921 INFO:teuthology.orchestra.run.smithi016.stdout:(40/133): libquadmath-11.4.1-3.el9.x86_64.rpm 548 kB/s | 195 kB 00:00 2024-08-17T02:43:06.923 INFO:teuthology.orchestra.run.smithi142.stdout:(77/133): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-08-17T02:43:06.963 INFO:teuthology.orchestra.run.smithi016.stdout:(41/133): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-08-17T02:43:06.973 INFO:teuthology.orchestra.run.smithi142.stdout:(78/133): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-08-17T02:43:06.998 INFO:teuthology.orchestra.run.smithi142.stdout:(79/133): python3-pytz-2021.1-5.el9.noarch.rpm 679 kB/s | 51 kB 00:00 2024-08-17T02:43:07.082 INFO:teuthology.orchestra.run.smithi142.stdout:(80/133): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-08-17T02:43:07.097 INFO:teuthology.orchestra.run.smithi016.stdout:(42/133): python3-cffi-1.14.5-5.el9.x86_64.rpm 675 kB/s | 253 kB 00:00 2024-08-17T02:43:07.107 INFO:teuthology.orchestra.run.smithi142.stdout:(81/133): python3-requests-oauthlib-1.3.0-12.el 402 kB/s | 54 kB 00:00 2024-08-17T02:43:07.141 INFO:teuthology.orchestra.run.smithi142.stdout:(82/133): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-08-17T02:43:07.204 INFO:teuthology.orchestra.run.smithi142.stdout:(83/133): socat-1.7.4.1-6.el9.x86_64.rpm 3.1 MB/s | 304 kB 00:00 2024-08-17T02:43:07.233 INFO:teuthology.orchestra.run.smithi142.stdout:(84/133): xfsprogs-devel-6.4.0-3.el9.x86_64.rpm 1.0 MB/s | 96 kB 00:00 2024-08-17T02:43:07.259 INFO:teuthology.orchestra.run.smithi142.stdout:(85/133): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-08-17T02:43:07.305 INFO:teuthology.orchestra.run.smithi016.stdout:(43/133): python3-ply-3.11-14.el9.noarch.rpm 342 kB/s | 106 kB 00:00 2024-08-17T02:43:07.317 INFO:teuthology.orchestra.run.smithi142.stdout:(86/133): zlib-devel-1.2.11-41.el9.x86_64.rpm 535 kB/s | 45 kB 00:00 2024-08-17T02:43:07.376 INFO:teuthology.orchestra.run.smithi142.stdout:(87/133): gperftools-libs-2.9.1-2.el9.x86_64.rp 2.6 MB/s | 309 kB 00:00 2024-08-17T02:43:07.401 INFO:teuthology.orchestra.run.smithi142.stdout:(88/133): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2024-08-17T02:43:07.414 INFO:teuthology.orchestra.run.smithi016.stdout:(44/133): python3-pycparser-2.20-6.el9.noarch.r 427 kB/s | 135 kB 00:00 2024-08-17T02:43:07.426 INFO:teuthology.orchestra.run.smithi142.stdout:(89/133): liboath-2.6.7-2.el9.x86_64.rpm 458 kB/s | 49 kB 00:00 2024-08-17T02:43:07.447 INFO:teuthology.orchestra.run.smithi016.stdout:(45/133): python3-cryptography-36.0.1-4.el9.x86 2.3 MB/s | 1.2 MB 00:00 2024-08-17T02:43:07.451 INFO:teuthology.orchestra.run.smithi142.stdout:(90/133): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-08-17T02:43:07.477 INFO:teuthology.orchestra.run.smithi142.stdout:(91/133): python3-cachetools-4.2.4-1.el9.noarch 1.3 MB/s | 32 kB 00:00 2024-08-17T02:43:07.502 INFO:teuthology.orchestra.run.smithi142.stdout:(92/133): python3-certifi-2023.05.07-4.el9.noar 573 kB/s | 14 kB 00:00 2024-08-17T02:43:07.556 INFO:teuthology.orchestra.run.smithi142.stdout:(93/133): python3-cheroot-8.6.0-4.el9.noarch.rp 3.1 MB/s | 172 kB 00:00 2024-08-17T02:43:07.586 INFO:teuthology.orchestra.run.smithi142.stdout:(94/133): python3-asyncssh-2.13.2-5.el9.noarch. 2.9 MB/s | 548 kB 00:00 2024-08-17T02:43:07.628 INFO:teuthology.orchestra.run.smithi142.stdout:(95/133): python3-google-auth-2.33.0-1.el9.noar 5.1 MB/s | 220 kB 00:00 2024-08-17T02:43:07.639 INFO:teuthology.orchestra.run.smithi016.stdout:(46/133): python3-requests-2.25.1-8.el9.noarch. 374 kB/s | 125 kB 00:00 2024-08-17T02:43:07.654 INFO:teuthology.orchestra.run.smithi142.stdout:(96/133): python3-cherrypy-18.6.1-2.el9.noarch. 3.6 MB/s | 358 kB 00:00 2024-08-17T02:43:07.679 INFO:teuthology.orchestra.run.smithi142.stdout:(97/133): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-08-17T02:43:07.704 INFO:teuthology.orchestra.run.smithi142.stdout:(98/133): python3-jaraco-classes-3.2.1-5.el9.no 353 kB/s | 18 kB 00:00 2024-08-17T02:43:07.723 INFO:teuthology.orchestra.run.smithi016.stdout:(47/133): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-08-17T02:43:07.729 INFO:teuthology.orchestra.run.smithi142.stdout:(99/133): python3-jaraco-collections-3.0.0-8.el 463 kB/s | 23 kB 00:00 2024-08-17T02:43:07.755 INFO:teuthology.orchestra.run.smithi142.stdout:(100/133): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-08-17T02:43:07.773 INFO:teuthology.orchestra.run.smithi016.stdout:(48/133): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-08-17T02:43:07.780 INFO:teuthology.orchestra.run.smithi142.stdout:(101/133): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-08-17T02:43:07.799 INFO:teuthology.orchestra.run.smithi016.stdout:(49/133): python3-urllib3-1.26.5-6.el9.noarch.r 559 kB/s | 215 kB 00:00 2024-08-17T02:43:07.805 INFO:teuthology.orchestra.run.smithi142.stdout:(102/133): python3-jwt+crypto-2.4.0-1.el9.noarc 181 kB/s | 9.0 kB 00:00 2024-08-17T02:43:07.831 INFO:teuthology.orchestra.run.smithi142.stdout:(103/133): python3-jwt-2.4.0-1.el9.noarch.rpm 812 kB/s | 41 kB 00:00 2024-08-17T02:43:07.840 INFO:teuthology.orchestra.run.smithi016.stdout:(50/133): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-08-17T02:43:07.856 INFO:teuthology.orchestra.run.smithi142.stdout:(104/133): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-17T02:43:07.914 INFO:teuthology.orchestra.run.smithi142.stdout:(105/133): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-08-17T02:43:07.924 INFO:teuthology.orchestra.run.smithi016.stdout:(51/133): flexiblas-3.0.4-8.el9.x86_64.rpm 262 kB/s | 33 kB 00:00 2024-08-17T02:43:07.940 INFO:teuthology.orchestra.run.smithi142.stdout:(106/133): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-17T02:43:07.966 INFO:teuthology.orchestra.run.smithi016.stdout:(52/133): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-08-17T02:43:07.973 INFO:teuthology.orchestra.run.smithi142.stdout:(107/133): python3-pecan-1.4.2-3.el9.noarch.rpm 8.1 MB/s | 272 kB 00:00 2024-08-17T02:43:07.991 INFO:teuthology.orchestra.run.smithi016.stdout:(53/133): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-08-17T02:43:08.007 INFO:teuthology.orchestra.run.smithi142.stdout:(108/133): python3-portend-3.1.0-2.el9.noarch.r 491 kB/s | 16 kB 00:00 2024-08-17T02:43:08.025 INFO:teuthology.orchestra.run.smithi016.stdout:(54/133): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-08-17T02:43:08.032 INFO:teuthology.orchestra.run.smithi142.stdout:(109/133): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-08-17T02:43:08.057 INFO:teuthology.orchestra.run.smithi142.stdout:(110/133): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-08-17T02:43:08.075 INFO:teuthology.orchestra.run.smithi016.stdout:(55/133): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-08-17T02:43:08.100 INFO:teuthology.orchestra.run.smithi142.stdout:(111/133): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-08-17T02:43:08.101 INFO:teuthology.orchestra.run.smithi016.stdout:(56/133): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-08-17T02:43:08.125 INFO:teuthology.orchestra.run.smithi142.stdout:(112/133): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-17T02:43:08.143 INFO:teuthology.orchestra.run.smithi016.stdout:(57/133): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-08-17T02:43:08.150 INFO:teuthology.orchestra.run.smithi142.stdout:(113/133): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-08-17T02:43:08.168 INFO:teuthology.orchestra.run.smithi016.stdout:(58/133): libpmemobj-1.12.1-1.el9.x86_64.rpm 908 kB/s | 160 kB 00:00 2024-08-17T02:43:08.175 INFO:teuthology.orchestra.run.smithi142.stdout:(114/133): python3-tempora-5.0.0-2.el9.noarch.r 709 kB/s | 36 kB 00:00 2024-08-17T02:43:08.201 INFO:teuthology.orchestra.run.smithi142.stdout:(115/133): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-08-17T02:43:08.201 INFO:teuthology.orchestra.run.smithi016.stdout:(59/133): libtool-2.4.6-46.el9.x86_64.rpm 5.6 MB/s | 581 kB 00:00 2024-08-17T02:43:08.227 INFO:teuthology.orchestra.run.smithi016.stdout:(60/133): libuuid-devel-2.37.4-18.el9.x86_64.rp 302 kB/s | 25 kB 00:00 2024-08-17T02:43:08.251 INFO:teuthology.orchestra.run.smithi142.stdout:(116/133): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-08-17T02:43:08.276 INFO:teuthology.orchestra.run.smithi142.stdout:(117/133): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-08-17T02:43:08.277 INFO:teuthology.orchestra.run.smithi016.stdout:(61/133): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-08-17T02:43:08.293 INFO:teuthology.orchestra.run.smithi142.stdout:(118/133): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-08-17T02:43:08.303 INFO:teuthology.orchestra.run.smithi016.stdout:(62/133): openblas-0.3.21-2.el9.x86_64.rpm 470 kB/s | 35 kB 00:00 2024-08-17T02:43:08.327 INFO:teuthology.orchestra.run.smithi142.stdout:(119/133): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-08-17T02:43:08.328 INFO:teuthology.orchestra.run.smithi016.stdout:(63/133): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-08-17T02:43:08.385 INFO:teuthology.orchestra.run.smithi142.stdout:(120/133): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-08-17T02:43:08.631 INFO:teuthology.orchestra.run.smithi142.stdout:(121/133): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-08-17T02:43:08.695 INFO:teuthology.orchestra.run.smithi016.stdout:(64/133): openssl-devel-3.2.2-2.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-08-17T02:43:08.719 INFO:teuthology.orchestra.run.smithi142.stdout:(122/133): librados2-17.2.7-1375.g2577e813.el9. 7.8 MB/s | 3.1 MB 00:00 2024-08-17T02:43:08.778 INFO:teuthology.orchestra.run.smithi142.stdout:(123/133): NetworkManager-1.48.2-2.el9.x86_64.r 16 MB/s | 2.3 MB 00:00 2024-08-17T02:43:08.779 INFO:teuthology.orchestra.run.smithi016.stdout:(65/133): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-08-17T02:43:08.837 INFO:teuthology.orchestra.run.smithi142.stdout:(124/133): librbd1-17.2.7-1375.g2577e813.el9.x8 6.5 MB/s | 2.9 MB 00:00 2024-08-17T02:43:08.845 INFO:teuthology.orchestra.run.smithi016.stdout:(66/133): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-08-17T02:43:08.862 INFO:teuthology.orchestra.run.smithi142.stdout:(125/133): NetworkManager-team-1.48.2-2.el9.x86 488 kB/s | 41 kB 00:00 2024-08-17T02:43:08.912 INFO:teuthology.orchestra.run.smithi142.stdout:(126/133): NetworkManager-libnm-1.48.2-2.el9.x8 9.4 MB/s | 1.8 MB 00:00 2024-08-17T02:43:08.921 INFO:teuthology.orchestra.run.smithi016.stdout:(67/133): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-08-17T02:43:08.938 INFO:teuthology.orchestra.run.smithi142.stdout:(127/133): elfutils-debuginfod-client-0.191-4.e 489 kB/s | 37 kB 00:00 2024-08-17T02:43:08.946 INFO:teuthology.orchestra.run.smithi016.stdout:(68/133): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-08-17T02:43:08.971 INFO:teuthology.orchestra.run.smithi016.stdout:(69/133): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-08-17T02:43:08.997 INFO:teuthology.orchestra.run.smithi142.stdout:(128/133): elfutils-libelf-0.191-4.el9.x86_64.r 2.4 MB/s | 207 kB 00:00 2024-08-17T02:43:09.005 INFO:teuthology.orchestra.run.smithi016.stdout:(70/133): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-08-17T02:43:09.021 INFO:teuthology.orchestra.run.smithi142.stdout:(129/133): NetworkManager-tui-1.48.2-2.el9.x86_ 1.3 MB/s | 247 kB 00:00 2024-08-17T02:43:09.030 INFO:teuthology.orchestra.run.smithi016.stdout:(71/133): python3-markupsafe-1.1.1-12.el9.x86_6 591 kB/s | 35 kB 00:00 2024-08-17T02:43:09.047 INFO:teuthology.orchestra.run.smithi142.stdout:(130/133): elfutils-libs-0.191-4.el9.x86_64.rpm 2.3 MB/s | 260 kB 00:00 2024-08-17T02:43:09.089 INFO:teuthology.orchestra.run.smithi016.stdout:(72/133): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-08-17T02:43:09.097 INFO:teuthology.orchestra.run.smithi142.stdout:(131/133): openssl-3.2.2-2.el9.x86_64.rpm 14 MB/s | 1.4 MB 00:00 2024-08-17T02:43:09.139 INFO:teuthology.orchestra.run.smithi142.stdout:(132/133): xfsprogs-6.4.0-3.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-08-17T02:43:09.147 INFO:teuthology.orchestra.run.smithi016.stdout:(73/133): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-08-17T02:43:09.198 INFO:teuthology.orchestra.run.smithi016.stdout:(74/133): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-17T02:43:09.248 INFO:teuthology.orchestra.run.smithi016.stdout:(75/133): python3-prettytable-0.7.2-27.el9.noar 848 kB/s | 42 kB 00:00 2024-08-17T02:43:09.306 INFO:teuthology.orchestra.run.smithi016.stdout:(76/133): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-08-17T02:43:09.336 INFO:teuthology.orchestra.run.smithi142.stdout:(133/133): openssl-libs-3.2.2-2.el9.x86_64.rpm 7.6 MB/s | 2.4 MB 00:00 2024-08-17T02:43:09.340 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-08-17T02:43:09.340 INFO:teuthology.orchestra.run.smithi142.stdout:Total 20 MB/s | 206 MB 00:10 2024-08-17T02:43:09.374 INFO:teuthology.orchestra.run.smithi016.stdout:(77/133): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-08-17T02:43:09.399 INFO:teuthology.orchestra.run.smithi016.stdout:(78/133): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-08-17T02:43:09.432 INFO:teuthology.orchestra.run.smithi016.stdout:(79/133): python3-pytz-2021.1-5.el9.noarch.rpm 877 kB/s | 51 kB 00:00 2024-08-17T02:43:09.458 INFO:teuthology.orchestra.run.smithi016.stdout:(80/133): python3-requests-oauthlib-1.3.0-12.el 914 kB/s | 54 kB 00:00 2024-08-17T02:43:09.517 INFO:teuthology.orchestra.run.smithi016.stdout:(81/133): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-08-17T02:43:09.575 INFO:teuthology.orchestra.run.smithi016.stdout:(82/133): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-08-17T02:43:09.642 INFO:teuthology.orchestra.run.smithi016.stdout:(83/133): openblas-openmp-0.3.21-2.el9.x86_64.r 3.5 MB/s | 4.7 MB 00:01 2024-08-17T02:43:09.659 INFO:teuthology.orchestra.run.smithi016.stdout:(84/133): xfsprogs-devel-6.4.0-3.el9.x86_64.rpm 1.1 MB/s | 96 kB 00:00 2024-08-17T02:43:09.701 INFO:teuthology.orchestra.run.smithi016.stdout:(85/133): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-08-17T02:43:09.735 INFO:teuthology.orchestra.run.smithi016.stdout:(86/133): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-08-17T02:43:09.835 INFO:teuthology.orchestra.run.smithi016.stdout:(87/133): liboath-2.6.7-2.el9.x86_64.rpm 486 kB/s | 49 kB 00:00 2024-08-17T02:43:09.860 INFO:teuthology.orchestra.run.smithi016.stdout:(88/133): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.9 MB/s | 309 kB 00:00 2024-08-17T02:43:09.886 INFO:teuthology.orchestra.run.smithi016.stdout:(89/133): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-08-17T02:43:09.911 INFO:teuthology.orchestra.run.smithi016.stdout:(90/133): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-08-17T02:43:09.936 INFO:teuthology.orchestra.run.smithi016.stdout:(91/133): python3-cachetools-4.2.4-1.el9.noarch 1.3 MB/s | 32 kB 00:00 2024-08-17T02:43:09.962 INFO:teuthology.orchestra.run.smithi016.stdout:(92/133): python3-certifi-2023.05.07-4.el9.noar 565 kB/s | 14 kB 00:00 2024-08-17T02:43:09.995 INFO:teuthology.orchestra.run.smithi016.stdout:(93/133): python3-cheroot-8.6.0-4.el9.noarch.rp 5.1 MB/s | 172 kB 00:00 2024-08-17T02:43:10.046 INFO:teuthology.orchestra.run.smithi016.stdout:(94/133): python3-asyncssh-2.13.2-5.el9.noarch. 2.9 MB/s | 548 kB 00:00 2024-08-17T02:43:10.079 INFO:teuthology.orchestra.run.smithi016.stdout:(95/133): python3-google-auth-2.33.0-1.el9.noar 6.5 MB/s | 220 kB 00:00 2024-08-17T02:43:10.104 INFO:teuthology.orchestra.run.smithi016.stdout:(96/133): python3-cherrypy-18.6.1-2.el9.noarch. 3.2 MB/s | 358 kB 00:00 2024-08-17T02:43:10.130 INFO:teuthology.orchestra.run.smithi016.stdout:(97/133): python3-jaraco-8.2.1-3.el9.noarch.rpm 212 kB/s | 11 kB 00:00 2024-08-17T02:43:10.155 INFO:teuthology.orchestra.run.smithi016.stdout:(98/133): python3-jaraco-classes-3.2.1-5.el9.no 716 kB/s | 18 kB 00:00 2024-08-17T02:43:10.180 INFO:teuthology.orchestra.run.smithi016.stdout:(99/133): python3-jaraco-collections-3.0.0-8.el 466 kB/s | 23 kB 00:00 2024-08-17T02:43:10.206 INFO:teuthology.orchestra.run.smithi016.stdout:(100/133): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-08-17T02:43:10.231 INFO:teuthology.orchestra.run.smithi016.stdout:(101/133): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-08-17T02:43:10.256 INFO:teuthology.orchestra.run.smithi016.stdout:(102/133): python3-jwt+crypto-2.4.0-1.el9.noarc 180 kB/s | 9.0 kB 00:00 2024-08-17T02:43:10.281 INFO:teuthology.orchestra.run.smithi016.stdout:(103/133): python3-jwt-2.4.0-1.el9.noarch.rpm 819 kB/s | 41 kB 00:00 2024-08-17T02:43:10.307 INFO:teuthology.orchestra.run.smithi016.stdout:(104/133): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-17T02:43:10.340 INFO:teuthology.orchestra.run.smithi016.stdout:(105/133): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-08-17T02:43:10.366 INFO:teuthology.orchestra.run.smithi016.stdout:(106/133): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-17T02:43:10.407 INFO:teuthology.orchestra.run.smithi016.stdout:(107/133): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-08-17T02:43:10.433 INFO:teuthology.orchestra.run.smithi016.stdout:(108/133): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-08-17T02:43:10.458 INFO:teuthology.orchestra.run.smithi016.stdout:(109/133): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-08-17T02:43:10.484 INFO:teuthology.orchestra.run.smithi016.stdout:(110/133): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-08-17T02:43:10.509 INFO:teuthology.orchestra.run.smithi016.stdout:(111/133): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-08-17T02:43:10.534 INFO:teuthology.orchestra.run.smithi016.stdout:(112/133): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-08-17T02:43:10.559 INFO:teuthology.orchestra.run.smithi016.stdout:(113/133): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-08-17T02:43:10.593 INFO:teuthology.orchestra.run.smithi016.stdout:(114/133): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-08-17T02:43:10.618 INFO:teuthology.orchestra.run.smithi016.stdout:(115/133): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-08-17T02:43:10.644 INFO:teuthology.orchestra.run.smithi016.stdout:(116/133): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-17T02:43:10.669 INFO:teuthology.orchestra.run.smithi016.stdout:(117/133): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-08-17T02:43:10.694 INFO:teuthology.orchestra.run.smithi016.stdout:(118/133): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-08-17T02:43:10.736 INFO:teuthology.orchestra.run.smithi016.stdout:(119/133): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-08-17T02:43:10.803 INFO:teuthology.orchestra.run.smithi016.stdout:(120/133): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-08-17T02:43:10.979 INFO:teuthology.orchestra.run.smithi016.stdout:(121/133): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-08-17T02:43:11.017 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-08-17T02:43:11.129 INFO:teuthology.orchestra.run.smithi016.stdout:(122/133): librados2-17.2.7-1375.g2577e813.el9. 7.8 MB/s | 3.1 MB 00:00 2024-08-17T02:43:11.179 INFO:teuthology.orchestra.run.smithi016.stdout:(123/133): librbd1-17.2.7-1375.g2577e813.el9.x8 7.8 MB/s | 2.9 MB 00:00 2024-08-17T02:43:11.221 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-08-17T02:43:11.222 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-08-17T02:43:11.471 INFO:teuthology.orchestra.run.smithi016.stdout:(124/133): NetworkManager-team-1.48.2-2.el9.x86 155 kB/s | 41 kB 00:00 2024-08-17T02:43:11.547 INFO:teuthology.orchestra.run.smithi016.stdout:(125/133): NetworkManager-1.48.2-2.el9.x86_64.r 4.1 MB/s | 2.3 MB 00:00 2024-08-17T02:43:11.705 INFO:teuthology.orchestra.run.smithi016.stdout:(126/133): NetworkManager-libnm-1.48.2-2.el9.x8 3.1 MB/s | 1.8 MB 00:00 2024-08-17T02:43:11.814 INFO:teuthology.orchestra.run.smithi016.stdout:(127/133): elfutils-debuginfod-client-0.191-4.e 155 kB/s | 37 kB 00:00 2024-08-17T02:43:11.906 INFO:teuthology.orchestra.run.smithi016.stdout:(128/133): NetworkManager-tui-1.48.2-2.el9.x86_ 570 kB/s | 247 kB 00:00 2024-08-17T02:43:12.048 INFO:teuthology.orchestra.run.smithi016.stdout:(129/133): elfutils-libelf-0.191-4.el9.x86_64.r 607 kB/s | 207 kB 00:00 2024-08-17T02:43:12.156 INFO:teuthology.orchestra.run.smithi016.stdout:(130/133): elfutils-libs-0.191-4.el9.x86_64.rpm 760 kB/s | 260 kB 00:00 2024-08-17T02:43:12.365 INFO:teuthology.orchestra.run.smithi016.stdout:(131/133): openssl-3.2.2-2.el9.x86_64.rpm 3.0 MB/s | 1.4 MB 00:00 2024-08-17T02:43:12.548 INFO:teuthology.orchestra.run.smithi016.stdout:(132/133): openssl-libs-3.2.2-2.el9.x86_64.rpm 4.8 MB/s | 2.4 MB 00:00 2024-08-17T02:43:12.623 INFO:teuthology.orchestra.run.smithi016.stdout:(133/133): xfsprogs-6.4.0-3.el9.x86_64.rpm 2.3 MB/s | 1.1 MB 00:00 2024-08-17T02:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-08-17T02:43:12.629 INFO:teuthology.orchestra.run.smithi016.stdout:Total 18 MB/s | 206 MB 00:11 2024-08-17T02:43:13.301 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-08-17T02:43:13.302 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-08-17T02:43:14.377 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-08-17T02:43:14.606 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-08-17T02:43:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-08-17T02:43:16.802 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-08-17T02:43:16.802 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-08-17T02:43:19.276 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-08-17T02:43:19.360 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/145 2024-08-17T02:43:19.411 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/145 2024-08-17T02:43:19.510 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/145 2024-08-17T02:43:19.554 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/145 2024-08-17T02:43:19.586 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/145 2024-08-17T02:43:19.645 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/145 2024-08-17T02:43:19.981 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/145 2024-08-17T02:43:20.078 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:17.2.7-1375.g2577e813.el9.x86_64 8/145 2024-08-17T02:43:20.239 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:17.2.7-1375.g2577e813.el9.x86_64 8/145 2024-08-17T02:43:20.262 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:17.2.7-1375.g2577e813.el9.x86_64 9/145 2024-08-17T02:43:20.366 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:17.2.7-1375.g2577e813.el9.x86_64 9/145 2024-08-17T02:43:20.833 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:17.2.7-1375.g2577e813.el9.x86_64 10/145 2024-08-17T02:43:20.872 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:17.2.7-1375.g2577e813.el9.x86_64 11/145 2024-08-17T02:43:20.959 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:17.2.7-1375.g2577e813.el9.x86_64 11/145 2024-08-17T02:43:21.031 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/145 2024-08-17T02:43:21.058 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:17.2.7-1375.g2577e813.el9.x86_64 13/145 2024-08-17T02:43:21.195 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1375.g2577e813.el9.x86_64 13/145 2024-08-17T02:43:21.224 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:17.2.7-1375.g2577e813.el9.x86 14/145 2024-08-17T02:43:21.378 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1375.g2577e813.el9.x86 14/145 2024-08-17T02:43:21.685 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 15/145 2024-08-17T02:43:21.903 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 16/145 2024-08-17T02:43:21.984 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 16/145 2024-08-17T02:43:22.091 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 17/145 2024-08-17T02:43:22.183 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 17/145 2024-08-17T02:43:22.473 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-08-17T02:43:22.515 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 17/145 2024-08-17T02:43:22.580 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/145 2024-08-17T02:43:22.587 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/145 2024-08-17T02:43:22.631 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/145 2024-08-17T02:43:22.634 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/145 2024-08-17T02:43:22.706 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/145 2024-08-17T02:43:22.716 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/145 2024-08-17T02:43:22.749 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/145 2024-08-17T02:43:22.775 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/145 2024-08-17T02:43:22.780 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/145 2024-08-17T02:43:22.809 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/145 2024-08-17T02:43:22.844 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/145 2024-08-17T02:43:22.898 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 23/145 2024-08-17T02:43:22.978 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 24/145 2024-08-17T02:43:23.012 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 25/145 2024-08-17T02:43:23.059 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 26/145 2024-08-17T02:43:23.139 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 27/145 2024-08-17T02:43:23.175 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:17.2.7-1375.g2577e813.el9. 28/145 2024-08-17T02:43:23.194 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/145 2024-08-17T02:43:23.221 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:17.2.7-1375.g2577e813.el 29/145 2024-08-17T02:43:23.289 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:17.2.7-1375.g2577e813.el9.x86_64 8/145 2024-08-17T02:43:23.340 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:17.2.7-1375.g2577e813.el9.x86_6 30/145 2024-08-17T02:43:23.427 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:17.2.7-1375.g2577e813.el9.x86_64 8/145 2024-08-17T02:43:23.456 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:17.2.7-1375.g2577e813.el9.x86_64 9/145 2024-08-17T02:43:23.515 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 31/145 2024-08-17T02:43:23.580 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:17.2.7-1375.g2577e813.el9.x86_64 9/145 2024-08-17T02:43:23.592 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 32/145 2024-08-17T02:43:24.083 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:17.2.7-1375.g2577e813.el9.x86_64 10/145 2024-08-17T02:43:24.116 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:17.2.7-1375.g2577e813.el9.x86_64 11/145 2024-08-17T02:43:24.202 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:17.2.7-1375.g2577e813.el9.x86_64 11/145 2024-08-17T02:43:24.246 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 33/145 2024-08-17T02:43:24.276 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/145 2024-08-17T02:43:24.302 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:17.2.7-1375.g2577e813.el9.x86_64 13/145 2024-08-17T02:43:24.374 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 34/145 2024-08-17T02:43:24.406 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 35/145 2024-08-17T02:43:24.425 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1375.g2577e813.el9.x86_64 13/145 2024-08-17T02:43:24.452 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:17.2.7-1375.g2577e813.el9.x86 14/145 2024-08-17T02:43:24.453 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 36/145 2024-08-17T02:43:24.497 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:17.2.7-1375.g2577e813.el9.x86_64 37/145 2024-08-17T02:43:24.535 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:17.2.7-1375.g2577e813.el9.x86_6 38/145 2024-08-17T02:43:24.579 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/145 2024-08-17T02:43:24.605 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1375.g2577e813.el9.x86 14/145 2024-08-17T02:43:24.618 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 40/145 2024-08-17T02:43:24.696 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/145 2024-08-17T02:43:24.907 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 15/145 2024-08-17T02:43:25.054 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/145 2024-08-17T02:43:25.095 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 43/145 2024-08-17T02:43:25.100 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 16/145 2024-08-17T02:43:25.131 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/145 2024-08-17T02:43:25.162 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 16/145 2024-08-17T02:43:25.188 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/145 2024-08-17T02:43:25.241 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/145 2024-08-17T02:43:25.277 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 17/145 2024-08-17T02:43:25.293 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/145 2024-08-17T02:43:25.370 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 17/145 2024-08-17T02:43:25.486 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/145 2024-08-17T02:43:25.541 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/145 2024-08-17T02:43:25.585 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/145 2024-08-17T02:43:25.699 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/145 2024-08-17T02:43:25.719 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 17/145 2024-08-17T02:43:25.741 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 52/145 2024-08-17T02:43:25.785 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/145 2024-08-17T02:43:25.815 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 54/145 2024-08-17T02:43:25.852 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 55/145 2024-08-17T02:43:25.890 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 56/145 2024-08-17T02:43:25.910 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/145 2024-08-17T02:43:25.927 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 57/145 2024-08-17T02:43:25.962 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 58/145 2024-08-17T02:43:25.964 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/145 2024-08-17T02:43:25.995 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 59/145 2024-08-17T02:43:26.048 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/145 2024-08-17T02:43:26.063 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 60/145 2024-08-17T02:43:26.096 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 61/145 2024-08-17T02:43:26.096 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/145 2024-08-17T02:43:26.129 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/145 2024-08-17T02:43:26.131 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 62/145 2024-08-17T02:43:26.180 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 63/145 2024-08-17T02:43:26.222 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 23/145 2024-08-17T02:43:26.286 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 24/145 2024-08-17T02:43:26.301 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-packaging-20.9-5.el9.noarch 64/145 2024-08-17T02:43:26.318 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 25/145 2024-08-17T02:43:26.366 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 26/145 2024-08-17T02:43:26.396 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 65/145 2024-08-17T02:43:26.456 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 27/145 2024-08-17T02:43:26.489 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:17.2.7-1375.g2577e813.el9. 28/145 2024-08-17T02:43:26.535 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:17.2.7-1375.g2577e813.el 29/145 2024-08-17T02:43:26.623 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:17.2.7-1375.g2577e813.el9.x86_6 30/145 2024-08-17T02:43:26.793 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 31/145 2024-08-17T02:43:26.857 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 32/145 2024-08-17T02:43:27.262 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 66/145 2024-08-17T02:43:27.323 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 67/145 2024-08-17T02:43:27.357 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 68/145 2024-08-17T02:43:27.486 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 69/145 2024-08-17T02:43:27.532 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 70/145 2024-08-17T02:43:27.537 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 33/145 2024-08-17T02:43:27.652 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 34/145 2024-08-17T02:43:27.754 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.21-2.el9.x86_64 71/145 2024-08-17T02:43:27.788 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 72/145 2024-08-17T02:43:27.882 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 73/145 2024-08-17T02:43:27.994 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 35/145 2024-08-17T02:43:28.052 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 36/145 2024-08-17T02:43:28.105 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:17.2.7-1375.g2577e813.el9.x86_64 37/145 2024-08-17T02:43:28.357 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:17.2.7-1375.g2577e813.el9.x86_6 38/145 2024-08-17T02:43:28.545 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/145 2024-08-17T02:43:28.584 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 40/145 2024-08-17T02:43:28.646 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/145 2024-08-17T02:43:28.667 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 74/145 2024-08-17T02:43:28.732 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/145 2024-08-17T02:43:28.776 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 43/145 2024-08-17T02:43:28.799 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 75/145 2024-08-17T02:43:28.813 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/145 2024-08-17T02:43:28.873 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/145 2024-08-17T02:43:28.925 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/145 2024-08-17T02:43:28.983 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/145 2024-08-17T02:43:29.165 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/145 2024-08-17T02:43:29.207 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/145 2024-08-17T02:43:29.243 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/145 2024-08-17T02:43:29.364 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/145 2024-08-17T02:43:29.410 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 52/145 2024-08-17T02:43:29.458 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/145 2024-08-17T02:43:29.497 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 54/145 2024-08-17T02:43:29.534 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 55/145 2024-08-17T02:43:29.572 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 56/145 2024-08-17T02:43:29.641 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 57/145 2024-08-17T02:43:29.678 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 58/145 2024-08-17T02:43:29.711 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 59/145 2024-08-17T02:43:29.746 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 60/145 2024-08-17T02:43:29.778 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 61/145 2024-08-17T02:43:29.813 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 62/145 2024-08-17T02:43:29.864 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 63/145 2024-08-17T02:43:29.993 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 64/145 2024-08-17T02:43:30.054 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 65/145 2024-08-17T02:43:30.368 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 76/145 2024-08-17T02:43:30.496 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 77/145 2024-08-17T02:43:30.537 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 78/145 2024-08-17T02:43:30.570 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 79/145 2024-08-17T02:43:30.887 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 80/145 2024-08-17T02:43:30.923 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:17.2.7-1375.g2577e813.el9.x86_64 81/145 2024-08-17T02:43:30.988 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 66/145 2024-08-17T02:43:31.022 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:17.2.7-1375.g2577e813.el9.x86_64 81/145 2024-08-17T02:43:31.046 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 67/145 2024-08-17T02:43:31.050 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:17.2.7-1375.g2577e813.el9.x86_64 82/145 2024-08-17T02:43:31.079 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 68/145 2024-08-17T02:43:31.198 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 69/145 2024-08-17T02:43:31.237 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 70/145 2024-08-17T02:43:31.477 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.21-2.el9.x86_64 71/145 2024-08-17T02:43:31.510 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 72/145 2024-08-17T02:43:31.616 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 73/145 2024-08-17T02:43:32.471 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 74/145 2024-08-17T02:43:32.627 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 75/145 2024-08-17T02:43:34.220 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:17.2.7-1375.g2577e813.el9.x86_64 83/145 2024-08-17T02:43:34.263 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:17.2.7-1375.g2577e813.el9.x86_64 83/145 2024-08-17T02:43:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 76/145 2024-08-17T02:43:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 77/145 2024-08-17T02:43:34.442 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 78/145 2024-08-17T02:43:34.509 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 79/145 2024-08-17T02:43:34.841 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 80/145 2024-08-17T02:43:34.859 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:17.2.7-1375.g2577e813.el9.x86_64 83/145 2024-08-17T02:43:34.887 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:17.2.7-1375.g2577e813.el9.x86_64 81/145 2024-08-17T02:43:34.899 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:17.2.7-1375.g2577e813.el9.x86_64 84/145 2024-08-17T02:43:35.004 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:17.2.7-1375.g2577e813.el9.x86_64 81/145 2024-08-17T02:43:35.030 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:17.2.7-1375.g2577e813.el9.x86_64 82/145 2024-08-17T02:43:35.041 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:17.2.7-1375.g2577e813.el9.x86_64 84/145 2024-08-17T02:43:35.041 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-17T02:43:35.041 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-17T02:43:35.041 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:43:35.084 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:17.2.7-1375.g2577e813.el9.x86_64 85/145 2024-08-17T02:43:37.736 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:17.2.7-1375.g2577e813.el9.x86_64 83/145 2024-08-17T02:43:37.777 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:17.2.7-1375.g2577e813.el9.x86_64 83/145 2024-08-17T02:43:38.398 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:17.2.7-1375.g2577e813.el9.x86_64 83/145 2024-08-17T02:43:38.447 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:17.2.7-1375.g2577e813.el9.x86_64 84/145 2024-08-17T02:43:38.585 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:17.2.7-1375.g2577e813.el9.x86_64 84/145 2024-08-17T02:43:38.586 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-17T02:43:38.586 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-17T02:43:38.586 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:43:38.623 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:17.2.7-1375.g2577e813.el9.x86_64 85/145 2024-08-17T02:43:46.854 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1375.g2577e813.el9.x86_64 85/145 2024-08-17T02:43:46.855 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-08-17T02:43:46.855 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-08-17T02:43:46.855 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-08-17T02:43:46.855 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-08-17T02:43:46.855 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-08-17T02:43:46.855 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2024-08-17T02:43:46.855 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-08-17T02:43:46.855 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:43:47.059 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:17.2.7-1375.g2577e813.el9.noarch 86/145 2024-08-17T02:43:47.095 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:17.2.7-1375.g2577e813.el9.noarch 86/145 2024-08-17T02:43:47.096 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:47.096 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-17T02:43:47.096 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:43:48.562 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:17.2.7-1375.g2577e813.el9.x86_64 87/145 2024-08-17T02:43:48.619 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:17.2.7-1375.g2577e813.el9.x86_64 87/145 2024-08-17T02:43:48.620 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:48.620 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-17T02:43:48.620 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-17T02:43:48.620 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-17T02:43:48.620 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:43:48.846 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:17.2.7-1375.g2577e813.el9.x86_64 88/145 2024-08-17T02:43:48.897 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:17.2.7-1375.g2577e813.el9.x86_64 88/145 2024-08-17T02:43:48.897 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:48.897 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-17T02:43:48.897 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-17T02:43:48.898 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-17T02:43:48.898 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:43:49.313 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:17.2.7-1375.g2577e813.el9.x86_64 89/145 2024-08-17T02:43:49.363 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:17.2.7-1375.g2577e813.el9.x86_64 89/145 2024-08-17T02:43:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-17T02:43:49.363 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-17T02:43:49.364 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-17T02:43:49.364 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:43:49.496 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.2-8.el9.noarch 90/145 2024-08-17T02:43:49.556 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 91/145 2024-08-17T02:43:49.620 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 92/145 2024-08-17T02:43:49.861 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 93/145 2024-08-17T02:43:49.916 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 94/145 2024-08-17T02:43:50.002 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 95/145 2024-08-17T02:43:50.118 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/145 2024-08-17T02:43:50.287 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 97/145 2024-08-17T02:43:50.354 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 98/145 2024-08-17T02:43:50.380 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 99/145 2024-08-17T02:43:50.416 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 100/145 2024-08-17T02:43:50.456 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 101/145 2024-08-17T02:43:50.482 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-16.el9.x86_64 102/145 2024-08-17T02:43:50.943 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:17.2.7-1375.g2577e813.el9.noarch 103/145 2024-08-17T02:43:50.973 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:17.2.7-1375.g2577e813.el9.noarch 103/145 2024-08-17T02:43:51.036 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1375.g2577e813.e 104/145 2024-08-17T02:43:51.097 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1375.g2577e813. 105/145 2024-08-17T02:43:51.128 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1375.g2577e813.el9.noa 106/145 2024-08-17T02:43:51.250 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1375.g2577e813.el9.x86_64 85/145 2024-08-17T02:43:51.250 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-08-17T02:43:51.250 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-08-17T02:43:51.250 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-08-17T02:43:51.250 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-08-17T02:43:51.250 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-08-17T02:43:51.250 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-08-17T02:43:51.250 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-08-17T02:43:51.250 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:43:51.451 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:17.2.7-1375.g2577e813.el9.noarch 86/145 2024-08-17T02:43:51.487 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:17.2.7-1375.g2577e813.el9.noarch 86/145 2024-08-17T02:43:51.487 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:51.488 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-17T02:43:51.488 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:43:52.270 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1375.g2577e813.el9.noa 106/145 2024-08-17T02:43:52.326 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1375.g2577e813.el9.n 107/145 2024-08-17T02:43:53.033 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:17.2.7-1375.g2577e813.el9.x86_64 87/145 2024-08-17T02:43:53.092 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:17.2.7-1375.g2577e813.el9.x86_64 87/145 2024-08-17T02:43:53.092 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:53.092 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-17T02:43:53.092 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-17T02:43:53.092 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-17T02:43:53.092 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:43:53.334 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:17.2.7-1375.g2577e813.el9.x86_64 88/145 2024-08-17T02:43:53.385 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:17.2.7-1375.g2577e813.el9.x86_64 88/145 2024-08-17T02:43:53.385 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:53.386 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-17T02:43:53.386 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-17T02:43:53.386 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-17T02:43:53.386 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:43:53.403 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1375.g2577e813.el9.n 107/145 2024-08-17T02:43:53.447 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1375.g257 108/145 2024-08-17T02:43:53.583 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1375.g257 108/145 2024-08-17T02:43:53.750 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1375.g2577e813.el 109/145 2024-08-17T02:43:53.783 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:17.2.7-1375.g2577e813.el9.x86_64 110/145 2024-08-17T02:43:53.810 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:17.2.7-1375.g2577e813.el9.x86_64 89/145 2024-08-17T02:43:53.834 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1375.g2577e813.el9.x86_64 110/145 2024-08-17T02:43:53.834 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:53.834 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-17T02:43:53.834 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-17T02:43:53.834 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-17T02:43:53.834 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:43:53.861 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:17.2.7-1375.g2577e813.el9.x86_64 89/145 2024-08-17T02:43:53.861 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:53.862 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-17T02:43:53.862 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-17T02:43:53.862 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-17T02:43:53.862 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:43:53.883 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:17.2.7-1375.g2577e813.el9.noarch 111/145 2024-08-17T02:43:53.909 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1375.g2577e813.el9.noarch 111/145 2024-08-17T02:43:53.995 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 90/145 2024-08-17T02:43:54.009 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:17.2.7-1375.g2577e813.el9.x86_64 112/145 2024-08-17T02:43:54.056 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 91/145 2024-08-17T02:43:54.070 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:17.2.7-1375.g2577e813.el9.x86_64 113/145 2024-08-17T02:43:54.129 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 92/145 2024-08-17T02:43:54.366 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 93/145 2024-08-17T02:43:54.423 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 94/145 2024-08-17T02:43:54.512 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 95/145 2024-08-17T02:43:54.623 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/145 2024-08-17T02:43:54.798 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 97/145 2024-08-17T02:43:54.853 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 98/145 2024-08-17T02:43:54.878 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 99/145 2024-08-17T02:43:54.915 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 100/145 2024-08-17T02:43:54.964 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 101/145 2024-08-17T02:43:54.989 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-16.el9.x86_64 102/145 2024-08-17T02:43:55.407 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-46.el9.x86_64 114/145 2024-08-17T02:43:55.430 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:17.2.7-1375.g2577e813.el9.noarch 103/145 2024-08-17T02:43:55.463 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:17.2.7-1375.g2577e813.el9.noarch 103/145 2024-08-17T02:43:55.476 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:17.2.7-1375.g2577e813.el9.x86_64 115/145 2024-08-17T02:43:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1375.g2577e813.e 104/145 2024-08-17T02:43:55.556 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1375.g2577e813. 105/145 2024-08-17T02:43:55.583 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1375.g2577e813.el9.x86_64 115/145 2024-08-17T02:43:55.583 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:55.583 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-17T02:43:55.583 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-17T02:43:55.584 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-17T02:43:55.584 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:43:55.585 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1375.g2577e813.el9.noa 106/145 2024-08-17T02:43:55.629 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1375.g2577e 116/145 2024-08-17T02:43:55.678 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1375.g2577e 116/145 2024-08-17T02:43:55.678 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:55.679 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-17T02:43:55.679 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:43:55.738 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 117/145 2024-08-17T02:43:55.788 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 117/145 2024-08-17T02:43:55.788 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:55.788 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-17T02:43:55.788 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-17T02:43:55.789 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-17T02:43:55.789 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:43:56.105 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 118/145 2024-08-17T02:43:56.157 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 118/145 2024-08-17T02:43:56.157 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:56.157 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-17T02:43:56.157 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-17T02:43:56.157 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-17T02:43:56.157 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:43:56.813 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1375.g2577e813.el9.noa 106/145 2024-08-17T02:43:56.854 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1375.g2577e813.el9.n 107/145 2024-08-17T02:43:58.008 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1375.g2577e813.el9.n 107/145 2024-08-17T02:43:58.029 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1375.g257 108/145 2024-08-17T02:43:58.172 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1375.g257 108/145 2024-08-17T02:43:58.360 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1375.g2577e813.el 109/145 2024-08-17T02:43:58.389 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:17.2.7-1375.g2577e813.el9.x86_64 110/145 2024-08-17T02:43:58.441 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1375.g2577e813.el9.x86_64 110/145 2024-08-17T02:43:58.441 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:43:58.442 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-17T02:43:58.442 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-17T02:43:58.442 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-17T02:43:58.442 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:43:58.490 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:17.2.7-1375.g2577e813.el9.noarch 111/145 2024-08-17T02:43:58.517 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1375.g2577e813.el9.noarch 111/145 2024-08-17T02:43:58.621 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:17.2.7-1375.g2577e813.el9.x86_64 112/145 2024-08-17T02:43:58.687 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:17.2.7-1375.g2577e813.el9.x86_64 113/145 2024-08-17T02:44:00.087 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-46.el9.x86_64 114/145 2024-08-17T02:44:00.126 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:17.2.7-1375.g2577e813.el9.x86_64 115/145 2024-08-17T02:44:00.240 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1375.g2577e813.el9.x86_64 115/145 2024-08-17T02:44:00.240 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:44:00.240 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-17T02:44:00.240 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-17T02:44:00.241 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-17T02:44:00.241 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:00.303 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1375.g2577e 116/145 2024-08-17T02:44:00.354 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1375.g2577e 116/145 2024-08-17T02:44:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:44:00.354 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-17T02:44:00.354 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:00.428 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 117/145 2024-08-17T02:44:00.480 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 117/145 2024-08-17T02:44:00.480 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:44:00.480 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-17T02:44:00.480 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-17T02:44:00.480 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-17T02:44:00.480 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:00.813 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 118/145 2024-08-17T02:44:00.826 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:17.2.7-1375.g2577e813.el9.x86_64 119/145 2024-08-17T02:44:00.864 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 118/145 2024-08-17T02:44:00.865 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-17T02:44:00.865 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-17T02:44:00.865 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-17T02:44:00.865 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-17T02:44:00.865 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:00.867 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:17.2.7-1375.g2577e813.el9.x86_64 120/145 2024-08-17T02:44:00.910 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:17.2.7-1375.g2577e813.el9.x86_64 121/145 2024-08-17T02:44:00.940 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 122/145 2024-08-17T02:44:00.999 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 123/145 2024-08-17T02:44:01.033 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:17.2.7-1375.g2577e813.el9.x86_ 124/145 2024-08-17T02:44:01.073 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 125/145 2024-08-17T02:44:01.105 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 126/145 2024-08-17T02:44:01.178 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 127/145 2024-08-17T02:44:02.455 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:17.2.7-1375.g2577e813.el9.noarch 128/145 2024-08-17T02:44:02.694 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 129/145 2024-08-17T02:44:02.781 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 130/145 2024-08-17T02:44:02.830 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 131/145 2024-08-17T02:44:02.941 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.4-9.el9.x86_64 132/145 2024-08-17T02:44:02.976 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.7.4-5.el9.x86_64 133/145 2024-08-17T02:44:03.031 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 134/145 2024-08-17T02:44:03.060 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 135/145 2024-08-17T02:44:03.096 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 136/145 2024-08-17T02:44:03.097 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 137/145 2024-08-17T02:44:03.213 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 137/145 2024-08-17T02:44:03.245 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 138/145 2024-08-17T02:44:03.245 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 139/145 2024-08-17T02:44:03.384 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 139/145 2024-08-17T02:44:03.421 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 140/145 2024-08-17T02:44:03.438 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 141/145 2024-08-17T02:44:03.438 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 141/145 2024-08-17T02:44:03.682 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 141/145 2024-08-17T02:44:03.682 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 142/145 2024-08-17T02:44:03.817 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 142/145 2024-08-17T02:44:03.850 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 143/145 2024-08-17T02:44:03.880 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 144/145 2024-08-17T02:44:03.956 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 145/145 2024-08-17T02:44:04.158 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 145/145 2024-08-17T02:44:04.159 INFO:teuthology.orchestra.run.smithi142.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-08-17T02:44:04.159 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:05.859 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:17.2.7-1375.g2577e813.el9.x86_64 119/145 2024-08-17T02:44:05.901 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:17.2.7-1375.g2577e813.el9.x86_64 120/145 2024-08-17T02:44:05.945 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:17.2.7-1375.g2577e813.el9.x86_64 121/145 2024-08-17T02:44:05.982 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 122/145 2024-08-17T02:44:06.016 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 123/145 2024-08-17T02:44:06.050 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:17.2.7-1375.g2577e813.el9.x86_ 124/145 2024-08-17T02:44:06.090 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 125/145 2024-08-17T02:44:06.121 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 126/145 2024-08-17T02:44:06.178 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 127/145 2024-08-17T02:44:07.585 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:17.2.7-1375.g2577e813.el9.noarch 128/145 2024-08-17T02:44:07.848 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 129/145 2024-08-17T02:44:07.964 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 130/145 2024-08-17T02:44:08.014 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 131/145 2024-08-17T02:44:08.123 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 132/145 2024-08-17T02:44:08.161 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 133/145 2024-08-17T02:44:08.194 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 134/145 2024-08-17T02:44:08.219 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 135/145 2024-08-17T02:44:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 136/145 2024-08-17T02:44:08.248 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 137/145 2024-08-17T02:44:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 137/145 2024-08-17T02:44:08.380 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 138/145 2024-08-17T02:44:08.381 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 139/145 2024-08-17T02:44:08.491 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 139/145 2024-08-17T02:44:08.515 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 140/145 2024-08-17T02:44:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 141/145 2024-08-17T02:44:08.534 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 141/145 2024-08-17T02:44:08.776 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 141/145 2024-08-17T02:44:08.776 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 142/145 2024-08-17T02:44:08.894 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 142/145 2024-08-17T02:44:08.926 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 143/145 2024-08-17T02:44:08.960 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 144/145 2024-08-17T02:44:09.037 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 145/145 2024-08-17T02:44:09.249 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 145/145 2024-08-17T02:44:09.249 INFO:teuthology.orchestra.run.smithi016.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-08-17T02:44:09.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:09.342 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:17.2.7-1375.g2577e813.el9.x86_64 1/145 2024-08-17T02:44:09.342 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:17.2.7-1375.g2577e813.el9.x86_64 2/145 2024-08-17T02:44:09.342 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:17.2.7-1375.g2577e813.el9.x86_64 3/145 2024-08-17T02:44:09.342 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:17.2.7-1375.g2577e813.el9.x86_64 4/145 2024-08-17T02:44:09.342 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1375.g2577e 5/145 2024-08-17T02:44:09.342 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:17.2.7-1375.g2577e813.el9.x86_64 6/145 2024-08-17T02:44:09.342 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:17.2.7-1375.g2577e813.el9.x86_64 7/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:17.2.7-1375.g2577e813.el9.x86_64 8/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:17.2.7-1375.g2577e813.el9.x86_64 9/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:17.2.7-1375.g2577e813.el9.x86_64 10/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:17.2.7-1375.g2577e813.el9.x86_64 11/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:17.2.7-1375.g2577e813.el9.x86_64 12/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 13/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:17.2.7-1375.g2577e813.el9.x86_ 14/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:17.2.7-1375.g2577e813.el9.x86_64 15/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:17.2.7-1375.g2577e813.el9.x86_64 16/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:17.2.7-1375.g2577e813.el9.x86_6 17/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:17.2.7-1375.g2577e813.el9.x86 18/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:17.2.7-1375.g2577e813.el9.x86_64 19/145 2024-08-17T02:44:09.343 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1375.g2577e813.el 20/145 2024-08-17T02:44:09.344 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:17.2.7-1375.g2577e813.el9. 21/145 2024-08-17T02:44:09.344 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:17.2.7-1375.g2577e813.el9.x86_6 22/145 2024-08-17T02:44:09.344 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:17.2.7-1375.g2577e813.el9.x86_64 23/145 2024-08-17T02:44:09.344 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:17.2.7-1375.g2577e813.el9.x86_64 24/145 2024-08-17T02:44:09.344 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:17.2.7-1375.g2577e813.el9.x86_64 25/145 2024-08-17T02:44:09.344 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:17.2.7-1375.g2577e813.el9.x86_64 26/145 2024-08-17T02:44:09.344 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 27/145 2024-08-17T02:44:09.344 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:17.2.7-1375.g2577e813.el9.x86_64 28/145 2024-08-17T02:44:09.344 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1375.g2577e813. 29/145 2024-08-17T02:44:09.357 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1375.g2577e813.el9.noa 30/145 2024-08-17T02:44:09.358 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1375.g2577e813.el9.n 31/145 2024-08-17T02:44:09.358 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1375.g257 32/145 2024-08-17T02:44:09.358 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1375.g2577e813.el 33/145 2024-08-17T02:44:09.358 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1375.g2577e813.el9.noarch 34/145 2024-08-17T02:44:09.358 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1375.g2577e813.e 35/145 2024-08-17T02:44:09.358 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:17.2.7-1375.g2577e813.el9.noarch 36/145 2024-08-17T02:44:09.359 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:17.2.7-1375.g2577e813.el9.noarch 37/145 2024-08-17T02:44:09.359 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:17.2.7-1375.g2577e813.el9.noarch 38/145 2024-08-17T02:44:09.359 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/145 2024-08-17T02:44:09.359 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 40/145 2024-08-17T02:44:09.359 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 41/145 2024-08-17T02:44:09.359 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/145 2024-08-17T02:44:09.360 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/145 2024-08-17T02:44:09.360 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/145 2024-08-17T02:44:09.360 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/145 2024-08-17T02:44:09.360 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/145 2024-08-17T02:44:09.360 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/145 2024-08-17T02:44:09.360 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/145 2024-08-17T02:44:09.360 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/145 2024-08-17T02:44:09.360 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 50/145 2024-08-17T02:44:09.360 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.4-9.el9.x86_64 51/145 2024-08-17T02:44:09.360 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 52/145 2024-08-17T02:44:09.361 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 53/145 2024-08-17T02:44:09.361 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 54/145 2024-08-17T02:44:09.361 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 55/145 2024-08-17T02:44:09.361 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 56/145 2024-08-17T02:44:09.361 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 57/145 2024-08-17T02:44:09.361 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 58/145 2024-08-17T02:44:09.361 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 59/145 2024-08-17T02:44:09.361 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 60/145 2024-08-17T02:44:09.361 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 61/145 2024-08-17T02:44:09.361 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 62/145 2024-08-17T02:44:09.362 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 63/145 2024-08-17T02:44:09.362 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 64/145 2024-08-17T02:44:09.362 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 65/145 2024-08-17T02:44:09.362 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/145 2024-08-17T02:44:09.362 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 67/145 2024-08-17T02:44:09.362 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 68/145 2024-08-17T02:44:09.362 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 69/145 2024-08-17T02:44:09.362 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 70/145 2024-08-17T02:44:09.362 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 71/145 2024-08-17T02:44:09.362 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 72/145 2024-08-17T02:44:09.362 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 73/145 2024-08-17T02:44:09.362 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 74/145 2024-08-17T02:44:09.363 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 75/145 2024-08-17T02:44:09.363 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 76/145 2024-08-17T02:44:09.363 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 77/145 2024-08-17T02:44:09.363 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/145 2024-08-17T02:44:09.363 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/145 2024-08-17T02:44:09.363 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/145 2024-08-17T02:44:09.363 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/145 2024-08-17T02:44:09.363 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/145 2024-08-17T02:44:09.364 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/145 2024-08-17T02:44:09.364 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 84/145 2024-08-17T02:44:09.364 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 85/145 2024-08-17T02:44:09.364 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 86/145 2024-08-17T02:44:09.364 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 87/145 2024-08-17T02:44:09.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 88/145 2024-08-17T02:44:09.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 89/145 2024-08-17T02:44:09.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 90/145 2024-08-17T02:44:09.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 91/145 2024-08-17T02:44:09.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 92/145 2024-08-17T02:44:09.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 93/145 2024-08-17T02:44:09.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 94/145 2024-08-17T02:44:09.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 95/145 2024-08-17T02:44:09.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 96/145 2024-08-17T02:44:09.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 97/145 2024-08-17T02:44:09.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 98/145 2024-08-17T02:44:09.365 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 99/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 100/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 101/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 102/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 103/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 104/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 105/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 106/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 107/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 108/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 109/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 110/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 111/145 2024-08-17T02:44:09.366 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 112/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 113/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 114/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 115/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 116/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 117/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 118/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 119/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 120/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 121/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:17.2.7-1375.g2577e813.el9.x86_64 122/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 123/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:17.2.7-1375.g2577e813.el9.x86_64 124/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 125/145 2024-08-17T02:44:09.367 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 126/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 127/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 128/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 129/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 130/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 131/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 132/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 133/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 134/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 135/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 136/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 137/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 138/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 139/145 2024-08-17T02:44:09.368 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 140/145 2024-08-17T02:44:09.369 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 141/145 2024-08-17T02:44:09.369 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 142/145 2024-08-17T02:44:09.369 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 143/145 2024-08-17T02:44:09.369 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 144/145 2024-08-17T02:44:11.400 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 145/145 2024-08-17T02:44:11.400 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-08-17T02:44:11.401 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.2-8.el9.noarch 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.7.4-5.el9.x86_64 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:11.402 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:11.403 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:11.403 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:11.403 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:11.403 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.403 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.403 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:11.403 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.403 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.403 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.403 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:11.403 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:11.403 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.404 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:11.404 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-17T02:44:11.404 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.4-9.el9.x86_64 2024-08-17T02:44:11.404 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-17T02:44:11.404 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-17T02:44:11.404 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-17T02:44:11.404 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-17T02:44:11.404 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-17T02:44:11.404 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-17T02:44:11.404 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.404 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.404 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.405 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-17T02:44:11.405 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-17T02:44:11.405 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-17T02:44:11.405 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-17T02:44:11.405 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-17T02:44:11.405 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.405 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.405 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-17T02:44:11.405 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.405 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-17T02:44:11.405 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-17T02:44:11.405 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-17T02:44:11.406 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-17T02:44:11.406 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-17T02:44:11.406 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-17T02:44:11.406 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-17T02:44:11.406 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-17T02:44:11.406 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-17T02:44:11.406 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-17T02:44:11.406 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-17T02:44:11.406 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-17T02:44:11.406 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.406 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.406 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.407 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-17T02:44:11.407 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-17T02:44:11.407 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-17T02:44:11.407 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-17T02:44:11.407 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-17T02:44:11.407 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-08-17T02:44:11.407 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-17T02:44:11.407 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-17T02:44:11.407 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-17T02:44:11.407 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-17T02:44:11.407 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-17T02:44:11.407 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-17T02:44:11.408 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-17T02:44:11.408 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-17T02:44:11.408 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-17T02:44:11.408 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-08-17T02:44:11.408 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-08-17T02:44:11.408 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-17T02:44:11.408 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-17T02:44:11.408 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-17T02:44:11.408 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-17T02:44:11.408 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-17T02:44:11.408 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-17T02:44:11.408 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-17T02:44:11.409 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-17T02:44:11.410 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-17T02:44:11.411 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-17T02:44:11.411 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-17T02:44:11.411 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-17T02:44:11.411 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.411 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.411 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:11.411 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-17T02:44:11.411 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-17T02:44:11.411 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-17T02:44:11.411 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-17T02:44:11.412 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-17T02:44:11.412 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:11.412 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-08-17T02:44:12.039 DEBUG:teuthology.parallel:result is None 2024-08-17T02:44:14.485 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:17.2.7-1375.g2577e813.el9.x86_64 1/145 2024-08-17T02:44:14.485 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:17.2.7-1375.g2577e813.el9.x86_64 2/145 2024-08-17T02:44:14.485 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:17.2.7-1375.g2577e813.el9.x86_64 3/145 2024-08-17T02:44:14.485 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:17.2.7-1375.g2577e813.el9.x86_64 4/145 2024-08-17T02:44:14.485 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1375.g2577e 5/145 2024-08-17T02:44:14.485 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:17.2.7-1375.g2577e813.el9.x86_64 6/145 2024-08-17T02:44:14.485 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:17.2.7-1375.g2577e813.el9.x86_64 7/145 2024-08-17T02:44:14.485 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:17.2.7-1375.g2577e813.el9.x86_64 8/145 2024-08-17T02:44:14.485 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:17.2.7-1375.g2577e813.el9.x86_64 9/145 2024-08-17T02:44:14.485 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:17.2.7-1375.g2577e813.el9.x86_64 10/145 2024-08-17T02:44:14.486 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:17.2.7-1375.g2577e813.el9.x86_64 11/145 2024-08-17T02:44:14.486 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:17.2.7-1375.g2577e813.el9.x86_64 12/145 2024-08-17T02:44:14.486 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 13/145 2024-08-17T02:44:14.486 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:17.2.7-1375.g2577e813.el9.x86_ 14/145 2024-08-17T02:44:14.486 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:17.2.7-1375.g2577e813.el9.x86_64 15/145 2024-08-17T02:44:14.486 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:17.2.7-1375.g2577e813.el9.x86_64 16/145 2024-08-17T02:44:14.486 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:17.2.7-1375.g2577e813.el9.x86_6 17/145 2024-08-17T02:44:14.486 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:17.2.7-1375.g2577e813.el9.x86 18/145 2024-08-17T02:44:14.486 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:17.2.7-1375.g2577e813.el9.x86_64 19/145 2024-08-17T02:44:14.486 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1375.g2577e813.el 20/145 2024-08-17T02:44:14.486 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:17.2.7-1375.g2577e813.el9. 21/145 2024-08-17T02:44:14.486 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:17.2.7-1375.g2577e813.el9.x86_6 22/145 2024-08-17T02:44:14.487 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:17.2.7-1375.g2577e813.el9.x86_64 23/145 2024-08-17T02:44:14.487 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:17.2.7-1375.g2577e813.el9.x86_64 24/145 2024-08-17T02:44:14.487 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:17.2.7-1375.g2577e813.el9.x86_64 25/145 2024-08-17T02:44:14.487 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:17.2.7-1375.g2577e813.el9.x86_64 26/145 2024-08-17T02:44:14.497 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 27/145 2024-08-17T02:44:14.497 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:17.2.7-1375.g2577e813.el9.x86_64 28/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1375.g2577e813. 29/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1375.g2577e813.el9.noa 30/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1375.g2577e813.el9.n 31/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1375.g257 32/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1375.g2577e813.el 33/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1375.g2577e813.el9.noarch 34/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1375.g2577e813.e 35/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:17.2.7-1375.g2577e813.el9.noarch 36/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:17.2.7-1375.g2577e813.el9.noarch 37/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:17.2.7-1375.g2577e813.el9.noarch 38/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 39/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 40/145 2024-08-17T02:44:14.498 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 41/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 42/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 43/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 44/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 45/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 46/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 47/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 48/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 49/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 50/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 51/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 52/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 53/145 2024-08-17T02:44:14.499 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 54/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 55/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 56/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 57/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 58/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 59/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 60/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 61/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 62/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 63/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 64/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 65/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/145 2024-08-17T02:44:14.500 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 67/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 68/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 69/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 70/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 71/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 72/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 73/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 74/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 75/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 76/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 77/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/145 2024-08-17T02:44:14.501 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 84/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 85/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 86/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 87/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 88/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 89/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 90/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 91/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 92/145 2024-08-17T02:44:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 93/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 94/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 95/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 96/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 97/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 98/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 99/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 100/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 101/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 102/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 103/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 104/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 105/145 2024-08-17T02:44:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 106/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 107/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 108/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 109/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 110/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 111/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 112/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 113/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 114/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 115/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 116/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 117/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 118/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 119/145 2024-08-17T02:44:14.504 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 120/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 121/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:17.2.7-1375.g2577e813.el9.x86_64 122/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 123/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:17.2.7-1375.g2577e813.el9.x86_64 124/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 125/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 126/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 127/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 128/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 129/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 130/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 131/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 132/145 2024-08-17T02:44:14.505 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 133/145 2024-08-17T02:44:14.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 134/145 2024-08-17T02:44:14.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 135/145 2024-08-17T02:44:14.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 136/145 2024-08-17T02:44:14.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 137/145 2024-08-17T02:44:14.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 138/145 2024-08-17T02:44:14.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 139/145 2024-08-17T02:44:14.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 140/145 2024-08-17T02:44:14.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 141/145 2024-08-17T02:44:14.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 142/145 2024-08-17T02:44:14.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 143/145 2024-08-17T02:44:14.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 144/145 2024-08-17T02:44:16.517 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 145/145 2024-08-17T02:44:16.518 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:16.518 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-08-17T02:44:16.518 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-08-17T02:44:16.518 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-08-17T02:44:16.518 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-08-17T02:44:16.518 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-08-17T02:44:16.519 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-08-17T02:44:16.519 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-08-17T02:44:16.519 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-08-17T02:44:16.519 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.519 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.519 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-08-17T02:44:16.520 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-08-17T02:44:16.520 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-08-17T02:44:16.520 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-08-17T02:44:16.520 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-08-17T02:44:16.520 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-08-17T02:44:16.520 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.521 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.521 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.521 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.521 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:16.521 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.521 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.522 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.522 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:16.522 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:16.522 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:16.522 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:16.522 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:16.523 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.523 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.523 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:16.523 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.523 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.523 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.524 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:16.524 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:16.525 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.525 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:17.2.7-1375.g2577e813.el9.noarch 2024-08-17T02:44:16.525 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-17T02:44:16.525 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-08-17T02:44:16.525 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-17T02:44:16.525 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-17T02:44:16.525 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-17T02:44:16.525 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-17T02:44:16.525 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-17T02:44:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-17T02:44:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-17T02:44:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-17T02:44:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-17T02:44:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-17T02:44:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-17T02:44:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.526 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-17T02:44:16.527 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-17T02:44:16.528 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-17T02:44:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-17T02:44:16.530 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-17T02:44:16.530 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-17T02:44:16.530 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-17T02:44:16.530 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-17T02:44:16.530 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-17T02:44:16.530 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-17T02:44:16.530 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-08-17T02:44:16.530 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-17T02:44:16.530 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-17T02:44:16.530 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-17T02:44:16.530 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-17T02:44:16.531 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:17.2.7-1375.g2577e813.el9.x86_64 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-17T02:44:16.532 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-17T02:44:16.533 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-17T02:44:16.533 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:16.533 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-08-17T02:44:17.275 DEBUG:teuthology.parallel:result is None 2024-08-17T02:44:17.276 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2577e8139c6b8786f9d6d6ea21b8da9244b9260e 2024-08-17T02:44:17.423 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-17T02:44:17.470 INFO:teuthology.orchestra.run.smithi016.stdout:17.2.7-1375.g2577e813.el9 2024-08-17T02:44:17.470 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1375.g2577e813.el9 2024-08-17T02:44:17.471 INFO:teuthology.task.install:The correct ceph version 17.2.7-1375.g2577e813 is installed. 2024-08-17T02:44:17.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2577e8139c6b8786f9d6d6ea21b8da9244b9260e 2024-08-17T02:44:17.609 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-17T02:44:17.659 INFO:teuthology.orchestra.run.smithi142.stdout:17.2.7-1375.g2577e813.el9 2024-08-17T02:44:17.660 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1375.g2577e813.el9 2024-08-17T02:44:17.660 INFO:teuthology.task.install:The correct ceph version 17.2.7-1375.g2577e813 is installed. 2024-08-17T02:44:17.662 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-17T02:44:17.662 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:17.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-17T02:44:17.727 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:17.727 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-17T02:44:17.790 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-17T02:44:17.790 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:17.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-08-17T02:44:17.848 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-17T02:44:17.953 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:17.953 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-08-17T02:44:18.013 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-17T02:44:18.110 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-17T02:44:18.111 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:18.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-17T02:44:18.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-17T02:44:18.267 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:18.267 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-17T02:44:18.323 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-17T02:44:18.418 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-17T02:44:18.418 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:18.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-08-17T02:44:18.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-17T02:44:18.569 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:18.569 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-08-17T02:44:18.625 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-17T02:44:18.721 INFO:teuthology.run_tasks:Running task ceph... 2024-08-17T02:44:18.815 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-08-17T02:44:18.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2024-08-17T02:44:18.820 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /var/log/ceph 2024-08-17T02:44:18.882 INFO:tasks.ceph:Disabling ceph logrotate... 2024-08-17T02:44:18.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-17T02:44:18.888 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/logrotate.d/ceph 2024-08-17T02:44:18.942 INFO:tasks.ceph:Creating extra log directories... 2024-08-17T02:44:18.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-17T02:44:18.946 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-08-17T02:44:19.007 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-08-17T02:44:19.007 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-08-17T02:44:19.008 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-08-17T02:44:19.008 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:19.008 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-08-17T02:44:19.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-08-17T02:44:19.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-08-17T02:44:19.274 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:19.274 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-08-17T02:44:19.309 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-08-17T02:44:19.478 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-08-17T02:44:19.539 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-08-17T02:44:19.539 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-08-17T02:44:19.540 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288', 'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7858288', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 25, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5995, 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-mchangir-mgr-volumes-print-thread-name-quincy', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e', 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMyjE/ovyyv8jkDimXUkeH3RLshdrtDmzbzTklszNSDNsX64CwAH69KFULc/On5kSIoI/8n7k24YFhLqaxnqvTY=', 'smithi142.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKo7qihD/ULkIowbEL69Jvq8IIu6heU6oXFnxNctDMbd3NEHXgkMVIA+k0bltnB+bLXxq+Kf6C142VdpMlWJ/Q4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', 'filesystem is online with fewer MDS than max_mds', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '2577e8139c6b8786f9d6d6ea21b8da9244b9260e', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '67db276914d29505c9707d04f3a3b8fb9d8aee4b', 'timestamp': '2024-08-17_01:10:27', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-08-17T02:44:19.540 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-17T02:44:19.545 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-08-17T02:44:19.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2024-08-17T02:44:19.621 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m0777 -- /var/run/ceph 2024-08-17T02:44:19.678 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:19.678 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-08-17T02:44:19.713 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-17T02:44:19.713 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-08-17T02:44:19.788 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-17T02:44:19.788 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:44:19.788 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-08-17T02:44:19.788 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:44:19.788 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-08-17T02:44:19.788 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-08-17 02:44:09.620022460 +0000 2024-08-17T02:44:19.788 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-08-17 02:41:47.074688776 +0000 2024-08-17T02:44:19.788 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-08-17 02:41:47.074688776 +0000 2024-08-17T02:44:19.788 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-08-17 02:41:47.074688776 +0000 2024-08-17T02:44:19.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-17T02:44:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-08-17T02:44:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-08-17T02:44:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000218253 s, 2.3 MB/s 2024-08-17T02:44:19.884 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-17T02:44:19.920 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-08-17T02:44:19.996 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-17T02:44:19.996 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:44:19.996 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-08-17T02:44:19.996 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:44:19.996 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-08-17T02:44:19.996 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-08-17 02:44:09.621022445 +0000 2024-08-17T02:44:19.996 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-08-17 02:41:47.531681839 +0000 2024-08-17T02:44:19.996 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-08-17 02:41:47.531681839 +0000 2024-08-17T02:44:19.996 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-08-17 02:41:47.531681839 +0000 2024-08-17T02:44:19.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-17T02:44:20.090 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-08-17T02:44:20.090 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-08-17T02:44:20.091 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000173962 s, 2.9 MB/s 2024-08-17T02:44:20.093 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-17T02:44:20.129 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-08-17T02:44:20.206 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-17T02:44:20.206 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:44:20.207 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-08-17T02:44:20.207 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:44:20.207 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-08-17T02:44:20.207 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-08-17 02:44:09.622022430 +0000 2024-08-17T02:44:20.207 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-08-17 02:41:48.010674568 +0000 2024-08-17T02:44:20.207 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-08-17 02:41:48.010674568 +0000 2024-08-17T02:44:20.207 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-08-17 02:41:48.010674568 +0000 2024-08-17T02:44:20.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-17T02:44:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-08-17T02:44:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-08-17T02:44:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000167293 s, 3.1 MB/s 2024-08-17T02:44:20.302 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-17T02:44:20.339 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-08-17T02:44:20.414 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-17T02:44:20.414 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:44:20.414 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-08-17T02:44:20.414 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:44:20.414 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-08-17T02:44:20.414 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-08-17 02:44:09.622022430 +0000 2024-08-17T02:44:20.415 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-08-17 02:41:48.495667205 +0000 2024-08-17T02:44:20.415 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-08-17 02:41:48.495667205 +0000 2024-08-17T02:44:20.415 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-08-17 02:41:48.495667205 +0000 2024-08-17T02:44:20.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-17T02:44:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-08-17T02:44:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-08-17T02:44:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000174037 s, 2.9 MB/s 2024-08-17T02:44:20.509 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-17T02:44:20.546 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-17T02:44:20.546 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:20.546 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-08-17T02:44:20.583 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-17T02:44:20.583 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-08-17T02:44:20.660 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-17T02:44:20.660 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:44:20.660 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 640 Links: 1 2024-08-17T02:44:20.660 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:44:20.660 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-08-17T02:44:20.660 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-08-17 02:44:04.510201394 +0000 2024-08-17T02:44:20.660 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-08-17 02:41:46.229352174 +0000 2024-08-17T02:44:20.660 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-08-17 02:41:46.229352174 +0000 2024-08-17T02:44:20.661 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-08-17 02:41:46.229352174 +0000 2024-08-17T02:44:20.661 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-17T02:44:20.752 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-08-17T02:44:20.753 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-08-17T02:44:20.753 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000280687 s, 1.8 MB/s 2024-08-17T02:44:20.755 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-17T02:44:20.791 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-08-17T02:44:20.866 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-17T02:44:20.866 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:44:20.866 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 658 Links: 1 2024-08-17T02:44:20.866 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:44:20.866 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-08-17T02:44:20.866 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-08-17 02:44:04.511201376 +0000 2024-08-17T02:44:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-08-17 02:41:46.712343294 +0000 2024-08-17T02:44:20.867 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-08-17 02:41:46.712343294 +0000 2024-08-17T02:44:20.867 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-08-17 02:41:46.712343294 +0000 2024-08-17T02:44:20.867 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-17T02:44:20.958 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-08-17T02:44:20.958 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-08-17T02:44:20.958 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000286467 s, 1.8 MB/s 2024-08-17T02:44:20.961 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-17T02:44:20.997 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-08-17T02:44:21.073 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-17T02:44:21.073 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:44:21.073 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 676 Links: 1 2024-08-17T02:44:21.074 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:44:21.074 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-08-17T02:44:21.074 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-08-17 02:44:04.511201376 +0000 2024-08-17T02:44:21.074 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-08-17 02:41:47.220333954 +0000 2024-08-17T02:44:21.074 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-08-17 02:41:47.220333954 +0000 2024-08-17T02:44:21.074 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-08-17 02:41:47.220333954 +0000 2024-08-17T02:44:21.074 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-17T02:44:21.166 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-08-17T02:44:21.166 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-08-17T02:44:21.166 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000282811 s, 1.8 MB/s 2024-08-17T02:44:21.168 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-17T02:44:21.205 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-08-17T02:44:21.281 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-17T02:44:21.281 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-17T02:44:21.281 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 694 Links: 1 2024-08-17T02:44:21.281 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-17T02:44:21.281 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-08-17T02:44:21.282 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-08-17 02:44:04.512201358 +0000 2024-08-17T02:44:21.282 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-08-17 02:41:47.700325129 +0000 2024-08-17T02:44:21.282 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-08-17 02:41:47.700325129 +0000 2024-08-17T02:44:21.282 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-08-17 02:41:47.700325129 +0000 2024-08-17T02:44:21.282 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-17T02:44:21.373 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-08-17T02:44:21.374 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-08-17T02:44:21.374 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000269326 s, 1.9 MB/s 2024-08-17T02:44:21.376 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-17T02:44:21.412 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-17T02:44:21.412 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi142.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-17T02:44:21.413 INFO:tasks.ceph:Generating config... 2024-08-17T02:44:21.414 INFO:tasks.ceph:[client] client mount timeout = 600 2024-08-17T02:44:21.414 INFO:tasks.ceph:[client] debug client = 20 2024-08-17T02:44:21.414 INFO:tasks.ceph:[client] debug ms = 1 2024-08-17T02:44:21.414 INFO:tasks.ceph:[client] pid file = 2024-08-17T02:44:21.414 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-08-17T02:44:21.414 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-08-17T02:44:21.414 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-08-17T02:44:21.414 INFO:tasks.ceph:[mds] debug mds = 20 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mds] debug ms = 1 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mds] mds debug frag = True 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mgr] debug client = 20 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-08-17T02:44:21.415 INFO:tasks.ceph:[mgr] debug ms = 1 2024-08-17T02:44:21.416 INFO:tasks.ceph:[mon] debug mon = 20 2024-08-17T02:44:21.416 INFO:tasks.ceph:[mon] debug ms = 1 2024-08-17T02:44:21.416 INFO:tasks.ceph:[mon] debug paxos = 20 2024-08-17T02:44:21.416 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-08-17T02:44:21.416 INFO:tasks.ceph:[osd] bdev async discard = True 2024-08-17T02:44:21.416 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-08-17T02:44:21.416 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-08-17T02:44:21.416 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-08-17T02:44:21.416 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-08-17T02:44:21.416 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-08-17T02:44:21.416 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-08-17T02:44:21.416 INFO:tasks.ceph:[osd] debug ms = 1 2024-08-17T02:44:21.417 INFO:tasks.ceph:[osd] debug osd = 20 2024-08-17T02:44:21.417 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-08-17T02:44:21.417 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-08-17T02:44:21.417 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-08-17T02:44:21.417 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-08-17T02:44:21.417 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-08-17T02:44:21.417 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-08-17T02:44:21.417 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-08-17T02:44:21.417 INFO:tasks.ceph:Setting up mon.a... 2024-08-17T02:44:21.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-08-17T02:44:21.507 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2024-08-17T02:44:21.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-08-17T02:44:21.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-17T02:44:21.649 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.b', '172.21.15.142'), ('mon.c', '[v2:172.21.15.142:3301,v1:172.21.15.142:6790]')] 2024-08-17T02:44:21.649 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-08-17T02:44:21.650 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:21.650 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-08-17T02:44:21.685 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --add b 172.21.15.142 --addv c '[v2:172.21.15.142:3301,v1:172.21.15.142:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:21.796 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:21.796 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid 4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778 2024-08-17T02:44:21.796 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = octopus 2024-08-17T02:44:21.796 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2024-08-17T02:44:21.796 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778 2024-08-17T02:44:21.796 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2024-08-17T02:44:21.796205+0000 2024-08-17T02:44:21.796 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-08-17T02:44:21.796205+0000 2024-08-17T02:44:21.796 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 15 (octopus) 2024-08-17T02:44:21.796 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2024-08-17T02:44:21.796 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-08-17T02:44:21.796 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-08-17T02:44:21.796 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] mon.c 2024-08-17T02:44:21.797 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-08-17T02:44:21.798 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-08-17T02:44:21.833 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778... 2024-08-17T02:44:21.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-17T02:44:21.878 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-08-17T02:44:21.971 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-08-17T02:44:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2024-08-17T02:44:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-17T02:44:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2024-08-17T02:44:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2024-08-17T02:44:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2024-08-17T02:44:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2024-08-17T02:44:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2024-08-17T02:44:21.972 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2024-08-17T02:44:21.973 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2024-08-17T02:44:21.974 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on skipped message = False 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.975 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2024-08-17T02:44:21.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: bdev async discard = True 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: bdev enable discard = True 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore allocator = bitmap 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2024-08-17T02:44:21.977 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 1/20 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 1/20 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 4/10 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.978 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2024-08-17T02:44:21.979 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-08-17T02:44:21.979 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2024-08-17T02:44:21.979 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-08-17T02:44:21.979 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-08-17T02:44:21.979 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-08-17T02:44:21.979 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-08-17T02:44:21.979 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2024-08-17T02:44:21.979 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-08-17T02:44:21.979 INFO:teuthology.orchestra.run.smithi016.stdout: debug finisher = 20 2024-08-17T02:44:21.979 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.979 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2024-08-17T02:44:21.979 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2024-08-17T02:44:21.980 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2024-08-17T02:44:21.981 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-08-17T02:44:21.982 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-08-17T02:44:21.983 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-08-17T02:44:21.983 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2024-08-17T02:44:21.983 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2024-08-17T02:44:21.983 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2024-08-17T02:44:21.983 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2024-08-17T02:44:21.983 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2024-08-17T02:44:21.983 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2024-08-17T02:44:21.983 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2024-08-17T02:44:21.983 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2024-08-17T02:44:21.983 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2024-08-17T02:44:21.983 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2024-08-17T02:44:21.983 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2024-08-17T02:44:21.984 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-08-17T02:44:21.984 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-08-17T02:44:21.984 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-08-17T02:44:21.984 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-08-17T02:44:21.984 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.984 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-08-17T02:44:21.984 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.984 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-08-17T02:44:21.984 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.984 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-08-17T02:44:21.984 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.985 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-08-17T02:44:21.986 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.16,172.21.15.142,[v2:172.21.15.142:3301,v1:172.21.15.142:6790]" 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on skipped message = False 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = 4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-08-17T02:44:21.987 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 600 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-08-17T02:44:21.988 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: bdev async discard = True 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: bdev enable discard = True 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore allocator = bitmap 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-08-17T02:44:21.989 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 1/20 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 1/20 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 4/10 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.990 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: debug finisher = 20 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-08-17T02:44:21.991 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-08-17T02:44:21.992 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: mon op complaint time = 120 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-08-17T02:44:21.993 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout: client mount timeout = 600 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout: debug client = 20 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = "" 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-08-17T02:44:21.994 INFO:teuthology.orchestra.run.smithi142.stdout:[mds] 2024-08-17T02:44:21.995 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds = 20 2024-08-17T02:44:21.995 INFO:teuthology.orchestra.run.smithi142.stdout: debug mds balancer = 20 2024-08-17T02:44:21.995 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-08-17T02:44:21.995 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug frag = True 2024-08-17T02:44:21.995 INFO:teuthology.orchestra.run.smithi142.stdout: mds debug scatterstat = True 2024-08-17T02:44:21.995 INFO:teuthology.orchestra.run.smithi142.stdout: mds op complaint time = 180 2024-08-17T02:44:21.995 INFO:teuthology.orchestra.run.smithi142.stdout: mds verify scatter = True 2024-08-17T02:44:21.995 INFO:teuthology.orchestra.run.smithi142.stdout: osd op complaint time = 180 2024-08-17T02:44:21.995 INFO:teuthology.orchestra.run.smithi142.stdout: rados mon op timeout = 900 2024-08-17T02:44:21.995 INFO:teuthology.orchestra.run.smithi142.stdout: rados osd op timeout = 900 2024-08-17T02:44:22.001 INFO:tasks.ceph:Creating admin key on mon.a... 2024-08-17T02:44:22.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-08-17T02:44:22.092 INFO:tasks.ceph:Copying monmap to all nodes... 2024-08-17T02:44:22.092 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:22.092 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-08-17T02:44:22.127 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:22.127 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-08-17T02:44:22.201 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:44:22.201 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:22.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-17T02:44:22.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-17T02:44:22.319 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:22.319 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:22.394 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:44:22.394 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:22.394 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.keyring 2024-08-17T02:44:22.394 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-08-17T02:44:22.473 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:22.474 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:22.549 INFO:tasks.ceph:Setting up mon nodes... 2024-08-17T02:44:22.549 INFO:tasks.ceph:Setting up mgr nodes... 2024-08-17T02:44:22.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-08-17T02:44:22.648 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-08-17T02:44:22.653 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-08-17T02:44:22.754 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-08-17T02:44:22.760 INFO:tasks.ceph:Setting up mds nodes... 2024-08-17T02:44:22.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-08-17T02:44:22.858 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-08-17T02:44:22.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-08-17T02:44:22.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-08-17T02:44:23.018 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-08-17T02:44:23.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-08-17T02:44:23.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-08-17T02:44:23.178 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-08-17T02:44:23.182 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-08-17T02:44:23.238 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-08-17T02:44:23.335 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-08-17T02:44:23.339 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-08-17T02:44:23.395 INFO:tasks.ceph_client:Setting up client nodes... 2024-08-17T02:44:23.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-17T02:44:23.473 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-08-17T02:44:23.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-08-17T02:44:23.575 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-08-17T02:44:23.602 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-08-17T02:44:23.676 INFO:teuthology.orchestra.run.smithi142.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-08-17T02:44:23.703 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-08-17T02:44:23.777 INFO:teuthology.orchestra.run.smithi142.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-08-17T02:44:23.804 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-08-17T02:44:23.804 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi142.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-08-17T02:44:23.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-08-17T02:44:23.861 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-17T02:44:23.861 INFO:tasks.ceph:role: osd.0 2024-08-17T02:44:23.861 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:44:23.861 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-17T02:44:23.917 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:44:23.917 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:44:23.917 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:44:23.917 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-17T02:44:23.918 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:44:23.918 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:44:23.918 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:44:23.918 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-17T02:44:23.918 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:44:23.918 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:44:23.950 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-08-17T02:44:23.953 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-08-17T02:44:23.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-08-17T02:44:24.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-08-17T02:44:24.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-08-17T02:44:24.155 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-17T02:44:24.155 INFO:tasks.ceph:role: osd.1 2024-08-17T02:44:24.155 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:44:24.156 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-17T02:44:24.210 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:44:24.211 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:44:24.211 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:44:24.211 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-17T02:44:24.211 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:44:24.211 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:44:24.211 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:44:24.211 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-17T02:44:24.211 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:44:24.211 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:44:24.243 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-08-17T02:44:24.247 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-08-17T02:44:24.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-08-17T02:44:24.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-08-17T02:44:24.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-08-17T02:44:24.447 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-17T02:44:24.447 INFO:tasks.ceph:role: osd.2 2024-08-17T02:44:24.447 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:44:24.447 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-17T02:44:24.503 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:44:24.503 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:44:24.503 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:44:24.503 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-17T02:44:24.503 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:44:24.504 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:44:24.504 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:44:24.504 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-17T02:44:24.504 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:44:24.504 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:44:24.538 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-08-17T02:44:24.542 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-08-17T02:44:24.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-08-17T02:44:24.624 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-08-17T02:44:24.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-08-17T02:44:24.749 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-08-17T02:44:24.749 INFO:tasks.ceph:role: osd.3 2024-08-17T02:44:24.749 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:44:24.749 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-17T02:44:24.806 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:44:24.806 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:44:24.807 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:44:24.807 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-17T02:44:24.807 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:44:24.807 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:44:24.807 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:44:24.807 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-17T02:44:24.807 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:44:24.807 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:44:24.840 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2024-08-17T02:44:24.844 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2024-08-17T02:44:24.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-08-17T02:44:24.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-08-17T02:44:24.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:25.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:25.092+0000 7f8e54c8b2c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-08-17T02:44:25.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:25.092+0000 7f8e54c8b2c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-08-17T02:44:25.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:25.092+0000 7f8e54c8b2c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-17T02:44:25.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:25.092+0000 7f8e54c8b2c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-17T02:44:25.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:25.093+0000 7f8e54c8b2c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-08-17T02:44:25.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:25.095+0000 7f8e54c8b2c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-08-17T02:44:25.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:25.638+0000 7f8e54c8b2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:26.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:26.409+0000 7f8e54c8b2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:27.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:27.180+0000 7f8e54c8b2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:28.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-08-17T02:44:28.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:28.158+0000 7f6636e0d2c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-08-17T02:44:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:28.159+0000 7f6636e0d2c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-08-17T02:44:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:28.159+0000 7f6636e0d2c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-17T02:44:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:28.159+0000 7f6636e0d2c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-17T02:44:28.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:28.159+0000 7f6636e0d2c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-08-17T02:44:28.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:28.161+0000 7f6636e0d2c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-08-17T02:44:28.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:28.703+0000 7f6636e0d2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:29.472+0000 7f6636e0d2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:30.243+0000 7f6636e0d2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:31.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-08-17T02:44:31.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:31.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:31.217+0000 7f56abd1d2c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-08-17T02:44:31.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:31.217+0000 7f56abd1d2c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-08-17T02:44:31.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:31.217+0000 7f56abd1d2c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-17T02:44:31.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:31.217+0000 7f56abd1d2c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-17T02:44:31.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:31.217+0000 7f56abd1d2c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-08-17T02:44:31.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:31.219+0000 7f56abd1d2c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-08-17T02:44:31.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:31.761+0000 7f56abd1d2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:32.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:32.533+0000 7f56abd1d2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:33.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:33.301+0000 7f56abd1d2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:34.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-08-17T02:44:34.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:34.283+0000 7f508af552c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-08-17T02:44:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:34.283+0000 7f508af552c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-08-17T02:44:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:34.283+0000 7f508af552c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-17T02:44:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:34.283+0000 7f508af552c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-17T02:44:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:34.283+0000 7f508af552c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-08-17T02:44:34.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:34.285+0000 7f508af552c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-08-17T02:44:34.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:34.828+0000 7f508af552c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:35.596+0000 7f508af552c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:36.366+0000 7f508af552c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:37.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-08-17T02:44:37.259 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-08-17T02:44:37.323 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-17T02:44:37.323 INFO:tasks.ceph:role: osd.4 2024-08-17T02:44:37.324 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:44:37.324 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-08-17T02:44:37.388 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:44:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:44:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:44:37.388 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-17T02:44:37.389 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:44:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:44:37.389 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:44:37.389 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-17T02:44:37.389 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:44:37.389 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:44:37.791 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-08-17T02:44:37.795 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-08-17T02:44:37.795 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-08-17T02:44:37.930 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-08-17T02:44:37.994 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-08-17T02:44:38.050 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-17T02:44:38.051 INFO:tasks.ceph:role: osd.5 2024-08-17T02:44:38.051 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:44:38.051 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-08-17T02:44:38.113 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:44:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:44:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:44:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-17T02:44:38.114 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:44:38.114 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:44:38.115 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:44:38.115 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-17T02:44:38.115 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:44:38.115 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:44:38.470 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-08-17T02:44:38.475 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-08-17T02:44:38.475 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-08-17T02:44:38.564 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-08-17T02:44:38.628 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-08-17T02:44:38.683 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-17T02:44:38.683 INFO:tasks.ceph:role: osd.6 2024-08-17T02:44:38.683 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:44:38.684 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-08-17T02:44:38.746 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:44:38.746 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:44:38.746 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:44:38.746 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-17T02:44:38.747 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:44:38.747 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:44:38.747 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:44:38.747 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-17T02:44:38.747 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:44:38.747 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:44:39.125 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-08-17T02:44:39.132 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-08-17T02:44:39.132 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-08-17T02:44:39.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-08-17T02:44:39.287 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-08-17T02:44:39.343 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-08-17T02:44:39.343 INFO:tasks.ceph:role: osd.7 2024-08-17T02:44:39.343 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:44:39.343 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-08-17T02:44:39.406 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-08-17T02:44:39.406 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-08-17T02:44:39.406 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-08-17T02:44:39.406 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-08-17T02:44:39.406 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-08-17T02:44:39.406 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-08-17T02:44:39.406 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-08-17T02:44:39.406 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-08-17T02:44:39.407 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-08-17T02:44:39.407 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-08-17T02:44:39.828 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-08-17T02:44:39.832 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-08-17T02:44:39.832 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-08-17T02:44:39.970 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-08-17T02:44:40.036 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:40.135 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:40.135+0000 7ff56ba162c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-08-17T02:44:40.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:40.137+0000 7ff56ba162c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-08-17T02:44:40.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:40.137+0000 7ff56ba162c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-17T02:44:40.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:40.137+0000 7ff56ba162c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-17T02:44:40.138 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:40.137+0000 7ff56ba162c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-08-17T02:44:40.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:40.142+0000 7ff56ba162c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-08-17T02:44:40.688 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:40.687+0000 7ff56ba162c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:41.460 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:41.460+0000 7ff56ba162c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:42.230 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:42.230+0000 7ff56ba162c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:43.062 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-08-17T02:44:43.120 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:43.203 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:43.202+0000 7f70f328b2c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-08-17T02:44:43.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:43.203+0000 7f70f328b2c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-08-17T02:44:43.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:43.203+0000 7f70f328b2c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-17T02:44:43.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:43.203+0000 7f70f328b2c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-17T02:44:43.204 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:43.203+0000 7f70f328b2c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-08-17T02:44:43.205 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:43.205+0000 7f70f328b2c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-08-17T02:44:43.759 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:43.758+0000 7f70f328b2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:44.539 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:44.539+0000 7f70f328b2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:45.310 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:45.309+0000 7f70f328b2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:46.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-08-17T02:44:46.217 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:46.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:46.301+0000 7f381a91d2c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-08-17T02:44:46.302 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:46.301+0000 7f381a91d2c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-08-17T02:44:46.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:46.302+0000 7f381a91d2c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-17T02:44:46.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:46.302+0000 7f381a91d2c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-17T02:44:46.303 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:46.302+0000 7f381a91d2c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-08-17T02:44:46.304 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:46.304+0000 7f381a91d2c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-08-17T02:44:46.860 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:46.859+0000 7f381a91d2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:47.632 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:47.632+0000 7f381a91d2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:48.396 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:48.395+0000 7f381a91d2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:49.218 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-08-17T02:44:49.274 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:49.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:49.358+0000 7fb30860b2c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-08-17T02:44:49.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:49.358+0000 7fb30860b2c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-08-17T02:44:49.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:49.358+0000 7fb30860b2c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-17T02:44:49.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:49.358+0000 7fb30860b2c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-17T02:44:49.359 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:49.358+0000 7fb30860b2c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-08-17T02:44:49.360 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:49.360+0000 7fb30860b2c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-08-17T02:44:49.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:44:49.546 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:44:49.903 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:49.903+0000 7fb30860b2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:50.669 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:50.668+0000 7fb30860b2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:51.443 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:44:51.443+0000 7fb30860b2c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-08-17T02:44:52.278 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-08-17T02:44:52.334 INFO:tasks.ceph:Reading keys from all nodes... 2024-08-17T02:44:52.334 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:52.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-08-17T02:44:52.397 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:52.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-08-17T02:44:52.496 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:52.496 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-08-17T02:44:52.592 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:52.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-08-17T02:44:52.689 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:52.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-08-17T02:44:52.784 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:52.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-08-17T02:44:52.879 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:52.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-08-17T02:44:52.974 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:52.974 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-08-17T02:44:53.032 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:53.032 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-08-17T02:44:53.129 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:53.129 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-08-17T02:44:53.223 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:53.224 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-08-17T02:44:53.319 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:53.319 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-08-17T02:44:53.414 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:53.415 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-08-17T02:44:53.511 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:53.511 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-08-17T02:44:53.606 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:53.607 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-08-17T02:44:53.645 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:44:53.646 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-08-17T02:44:53.723 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:53.723 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-08-17T02:44:53.758 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:44:53.758 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-08-17T02:44:53.833 INFO:tasks.ceph:Adding keys to all mons... 2024-08-17T02:44:53.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-17T02:44:53.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo tee -a /etc/ceph/ceph.keyring 2024-08-17T02:44:53.888 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.y] 2024-08-17T02:44:53.888 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGDsBm5S2qJhAAbhnTqOJjI5AwT0ET6q/GRg== 2024-08-17T02:44:53.888 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2024-08-17T02:44:53.888 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGDsBmqW8wMxAAq+fTCp8vlCJN6+cR/DX7EA== 2024-08-17T02:44:53.888 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2024-08-17T02:44:53.888 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCHDsBmMhIjARAAmNH2mDnE7u+gPFZBvH0YOA== 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCJDsBmDc6YBRAAr1Dv8OP3NemJBuGlEKic+w== 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCMDsBmhIWNCRAAZazsazPPVNJkGx16uvsmhg== 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCPDsBm7tEHDRAAfoe57cHfMYzOyluVRl7OWg== 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCSDsBmhBH0EBAAy2nZuWtACHGHNBXHSs0x5Q== 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGDsBm3fn6LBAAMMskmYZc6ASOkn+stKKllw== 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCHDsBmY02mChAAwbxC+Xnlh+vHDhbrKl0MHg== 2024-08-17T02:44:53.889 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCHDsBmQav9ExAAlAXMmqzLG2nF2c8Jo4pMzg== 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCYDsBmUzYaCBAAD1a9iSf7v8jH3YImMPUX2A== 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCbDsBmQjYkDBAAdlCM8QfcMLK5dBRR1LMIIQ== 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCeDsBmsLwEEhAAMIvs0maEzEEHCxBwO0Ae/A== 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQChDsBmuGZmFRAAIho6JxP3NTmgOsuCmH3x0w== 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCHDsBmLsQ9HBAA8Lj9LuvNWPNmRDuhWQDMpQ== 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout:[client.2] 2024-08-17T02:44:53.890 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCHDsBmNQ9TIhAACLZVzo0XXehLBoawb5yqzw== 2024-08-17T02:44:53.891 INFO:teuthology.orchestra.run.smithi016.stdout:[client.1] 2024-08-17T02:44:53.891 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCHDsBmAJZaKBAALuHMnu/+CvfwswA+EC/zjg== 2024-08-17T02:44:53.893 INFO:teuthology.orchestra.run.smithi016.stdout:[client.3] 2024-08-17T02:44:53.894 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCHDsBmWZpeLhAAozMeV17bYm3xQMnkbof5vQ== 2024-08-17T02:44:53.925 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr.y] 2024-08-17T02:44:53.925 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCGDsBm5S2qJhAAbhnTqOJjI5AwT0ET6q/GRg== 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.a] 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCGDsBmqW8wMxAAq+fTCp8vlCJN6+cR/DX7EA== 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.c] 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCHDsBmMhIjARAAmNH2mDnE7u+gPFZBvH0YOA== 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.0] 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCJDsBmDc6YBRAAr1Dv8OP3NemJBuGlEKic+w== 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.1] 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCMDsBmhIWNCRAAZazsazPPVNJkGx16uvsmhg== 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.2] 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCPDsBm7tEHDRAAfoe57cHfMYzOyluVRl7OWg== 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.3] 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCSDsBmhBH0EBAAy2nZuWtACHGHNBXHSs0x5Q== 2024-08-17T02:44:53.926 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr.x] 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCGDsBm3fn6LBAAMMskmYZc6ASOkn+stKKllw== 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.b] 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCHDsBmY02mChAAwbxC+Xnlh+vHDhbrKl0MHg== 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout:[mds.d] 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCHDsBmQav9ExAAlAXMmqzLG2nF2c8Jo4pMzg== 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.4] 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCYDsBmUzYaCBAAD1a9iSf7v8jH3YImMPUX2A== 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.5] 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCbDsBmQjYkDBAAdlCM8QfcMLK5dBRR1LMIIQ== 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.6] 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCeDsBmsLwEEhAAMIvs0maEzEEHCxBwO0Ae/A== 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout:[osd.7] 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQChDsBmuGZmFRAAIho6JxP3NTmgOsuCmH3x0w== 2024-08-17T02:44:53.927 INFO:teuthology.orchestra.run.smithi142.stdout:[client.0] 2024-08-17T02:44:53.928 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCHDsBmLsQ9HBAA8Lj9LuvNWPNmRDuhWQDMpQ== 2024-08-17T02:44:53.928 INFO:teuthology.orchestra.run.smithi142.stdout:[client.2] 2024-08-17T02:44:53.928 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCHDsBmNQ9TIhAACLZVzo0XXehLBoawb5yqzw== 2024-08-17T02:44:53.928 INFO:teuthology.orchestra.run.smithi142.stdout:[client.1] 2024-08-17T02:44:53.928 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCHDsBmAJZaKBAALuHMnu/+CvfwswA+EC/zjg== 2024-08-17T02:44:53.928 INFO:teuthology.orchestra.run.smithi142.stdout:[client.3] 2024-08-17T02:44:53.928 INFO:teuthology.orchestra.run.smithi142.stdout: key = AQCHDsBmWZpeLhAAozMeV17bYm3xQMnkbof5vQ== 2024-08-17T02:44:53.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-17T02:44:53.939 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-17T02:44:54.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:44:54.060 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:44:54.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:44:54.153 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:44:54.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:54.246 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:54.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:54.339 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:54.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:54.429 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:54.516 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:54.520 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:54.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-17T02:44:54.609 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-08-17T02:44:54.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:44:54.699 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:44:54.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:44:54.792 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-08-17T02:44:54.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:54.884 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:54.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:54.975 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:55.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:55.066 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:55.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:55.156 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-08-17T02:44:55.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:44:55.247 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:44:55.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:44:55.340 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:44:55.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:44:55.433 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:44:55.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:44:55.526 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-08-17T02:44:55.618 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-08-17T02:44:55.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-08-17T02:44:55.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-17T02:44:55.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-08-17T02:44:55.831 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-08-17T02:44:55.887 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-17T02:44:55.987 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-08-17T02:44:56.042 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-08-17T02:44:56.097 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-08-17T02:44:56.197 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-08-17T02:44:56.252 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:56.256 DEBUG:teuthology.orchestra.run.smithi142:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-08-17T02:44:56.291 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-08-17T02:44:56.291 INFO:tasks.ceph.mon.a:Restarting daemon 2024-08-17T02:44:56.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-08-17T02:44:56.333 INFO:tasks.ceph.mon.a:Started 2024-08-17T02:44:56.333 INFO:tasks.ceph.mon.b:Restarting daemon 2024-08-17T02:44:56.333 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-08-17T02:44:56.337 INFO:tasks.ceph.mon.b:Started 2024-08-17T02:44:56.337 INFO:tasks.ceph.mon.c:Restarting daemon 2024-08-17T02:44:56.337 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-08-17T02:44:56.341 INFO:tasks.ceph.mon.c:Started 2024-08-17T02:44:56.341 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-08-17T02:44:56.341 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-08-17T02:44:56.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-08-17T02:44:56.345 INFO:tasks.ceph.mgr.y:Started 2024-08-17T02:44:56.345 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-08-17T02:44:56.346 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-08-17T02:44:56.353 INFO:tasks.ceph.mgr.x:Started 2024-08-17T02:44:56.353 INFO:tasks.ceph:Setting crush tunables to default 2024-08-17T02:44:56.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2024-08-17T02:44:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.505+0000 7f7fae200640 1 Processor -- start 2024-08-17T02:44:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.505+0000 7f7fae200640 1 -- start start 2024-08-17T02:44:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7fae200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:44:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7fae200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:44:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7fae200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa8158400 con 0x7f7fa8155980 2024-08-17T02:44:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7fae200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa80aff40 con 0x7f7fa80a54d0 2024-08-17T02:44:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7fae200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81577f0 con 0x7f7fa80af580 2024-08-17T02:44:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7f9fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:44:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7fa7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:44:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7f9fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:60996/0 (socket says 172.21.15.16:60996) 2024-08-17T02:44:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2222888978 learned_addr learned my addr 172.21.15.16:0/2222888978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:44:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 msgr2=0x7f7fa8155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-17T02:44:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 msgr2=0x7f7fa8155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:44:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:44:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-17T02:44:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7fa7e00640 1 -- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 msgr2=0x7f7fa80afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-17T02:44:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7fa7e00640 1 -- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 msgr2=0x7f7fa80afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:44:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:44:56.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.507+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-17T02:44:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.709+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:44:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.709+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 msgr2=0x7f7fa8155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-17T02:44:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.709+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 msgr2=0x7f7fa8155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:44:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.709+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:44:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.709+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-17T02:44:56.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.709+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:44:56.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.709+0000 7f7fa7e00640 1 -- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 msgr2=0x7f7fa80afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-17T02:44:56.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.709+0000 7f7fa7e00640 1 -- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 msgr2=0x7f7fa80afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:44:56.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.709+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:44:56.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:56.709+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-17T02:44:57.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.110+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:44:57.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.110+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:44:57.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.110+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 msgr2=0x7f7fa8155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-17T02:44:57.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.110+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 msgr2=0x7f7fa8155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:44:57.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.110+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:44:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.110+0000 7f7fa7e00640 1 -- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 msgr2=0x7f7fa80afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-17T02:44:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.110+0000 7f7fa7e00640 1 -- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 msgr2=0x7f7fa80afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:44:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.110+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:44:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.110+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-17T02:44:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.110+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-17T02:44:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.911+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:44:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.911+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:44:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.911+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 msgr2=0x7f7fa8155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-17T02:44:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.911+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 msgr2=0x7f7fa8155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:44:57.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.911+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:44:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.912+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-17T02:44:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.912+0000 7f7fa7e00640 1 -- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 msgr2=0x7f7fa80afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-08-17T02:44:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.912+0000 7f7fa7e00640 1 -- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 msgr2=0x7f7fa80afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:44:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.912+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:44:57.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:57.912+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-17T02:44:59.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.507+0000 7f7fa5600640 1 -- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 msgr2=0x7f7fa80afa00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:44:59.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.507+0000 7f7fa5600640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7fa80af580 0x7f7fa80afa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:44:59.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.507+0000 7f7fa5600640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 msgr2=0x7f7fa8155d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:44:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.507+0000 7f7fa5600640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.142:3300/0 conn(0x7f7fa80a54d0 0x7f7fa8155d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:44:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.507+0000 7f7fa5600640 1 -- 172.21.15.16:0/2222888978 >> v1:172.21.15.16:6789/0 conn(0x7f7fa8155980 legacy=0x7f7fa8156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:44:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.507+0000 7f7fa5600640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:44:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.507+0000 7f7fa5600640 1 -- 172.21.15.16:0/2222888978 --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa8157ad0 con 0x7f7f84006120 2024-08-17T02:44:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.507+0000 7f7fa5600640 1 -- 172.21.15.16:0/2222888978 --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa80aff40 con 0x7f7f84003680 2024-08-17T02:44:59.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.508+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:44:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.508+0000 7f7fa7400640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 msgr2=0x7f7f84005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-17T02:44:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.508+0000 7f7fa7400640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 msgr2=0x7f7f84005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:44:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.508+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:44:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.508+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-17T02:44:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.709+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:44:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.709+0000 7f7fa7400640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 msgr2=0x7f7f84005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-17T02:44:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.709+0000 7f7fa7400640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 msgr2=0x7f7f84005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:44:59.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.709+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:44:59.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:44:59.709+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-17T02:45:00.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:00.110+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:00.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:00.110+0000 7f7fa7400640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 msgr2=0x7f7f84005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-17T02:45:00.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:00.110+0000 7f7fa7400640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 msgr2=0x7f7f84005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:45:00.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:00.110+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:45:00.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:00.110+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-17T02:45:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:00.912+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:00.912+0000 7f7fa7400640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 msgr2=0x7f7f84005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-08-17T02:45:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:00.912+0000 7f7fa7400640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 msgr2=0x7f7f84005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-08-17T02:45:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:00.912+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-08-17T02:45:00.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:00.912+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-17T02:45:01.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.714+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205855366 0 0) 0x7f7fa8157ad0 con 0x7f7f84006120 2024-08-17T02:45:01.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.714+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f8c0034e0 con 0x7f7f84006120 2024-08-17T02:45:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2158761219 0 0) 0x7f7f8c0034e0 con 0x7f7f84006120 2024-08-17T02:45:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 msgr2=0x7f7f84005af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:45:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa6a00640 1 --2- 172.21.15.16:0/2222888978 >> v2:172.21.15.16:3300/0 conn(0x7f7f84003680 0x7f7f84005af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 --> v1:172.21.15.142:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8400aeb0 con 0x7f7f84006120 2024-08-17T02:45:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 <== mon.1 v1:172.21.15.142:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7f7f9000c2f0 con 0x7f7f84006120 2024-08-17T02:45:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 >> v1:172.21.15.142:6789/0 conn(0x7f7f84006120 legacy=0x7f7f840085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa7e00640 1 -- 172.21.15.16:0/2222888978 reap_dead start 2024-08-17T02:45:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa6a00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f8c003a20 0x7f7f8c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa6a00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 0x7f7f8c004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa6a00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c005330 0x7f7f8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80aff40 con 0x7f7f8c004780 2024-08-17T02:45:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c00c260 con 0x7f7f8c005330 2024-08-17T02:45:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.717+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c00c420 con 0x7f7f8c003a20 2024-08-17T02:45:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.718+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 0x7f7f8c004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.718+0000 7f7fa7400640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f8c003a20 0x7f7f8c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.718+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c005330 0x7f7f8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:01.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.718+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f8c003a20 msgr2=0x7f7f8c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:01.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.718+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f8c003a20 0x7f7f8c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:01.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.718+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c005330 msgr2=0x7f7f8c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:01.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.718+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c005330 0x7f7f8c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:01.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.718+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2222888978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c0034e0 con 0x7f7f8c004780 2024-08-17T02:45:01.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.719+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 0x7f7f8c004bf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f90006fd0 tx=0x7f7f90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467885d58b543f19 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:01.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.720+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9002dab0 con 0x7f7f8c004780 2024-08-17T02:45:01.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.720+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9000c040 con 0x7f7f8c004780 2024-08-17T02:45:01.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.720+0000 7f7fa6a00640 1 -- 172.21.15.16:0/2222888978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9002dcf0 con 0x7f7f8c004780 2024-08-17T02:45:01.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.721+0000 7f7fae200640 1 -- 172.21.15.16:0/2222888978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 msgr2=0x7f7f8c004bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:01.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.721+0000 7f7fae200640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 0x7f7f8c004bf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7f90006fd0 tx=0x7f7f90008040 comp rx=0 tx=0).stop 2024-08-17T02:45:01.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.721+0000 7f7fae200640 1 -- 172.21.15.16:0/2222888978 shutdown_connections 2024-08-17T02:45:01.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.721+0000 7f7fae200640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c005330 0x7f7f8c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:01.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.721+0000 7f7fae200640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 0x7f7f8c004bf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:01.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.721+0000 7f7fae200640 1 --2- 172.21.15.16:0/2222888978 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f8c003a20 0x7f7f8c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.721+0000 7f7fae200640 1 -- 172.21.15.16:0/2222888978 >> 172.21.15.16:0/2222888978 conn(0x7f7fa8067750 msgr2=0x7f7fa80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.721+0000 7f7fae200640 1 -- 172.21.15.16:0/2222888978 shutdown_connections 2024-08-17T02:45:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.721+0000 7f7fae200640 1 -- 172.21.15.16:0/2222888978 wait complete. 2024-08-17T02:45:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.723+0000 7f7fae200640 1 Processor -- start 2024-08-17T02:45:01.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.723+0000 7f7fae200640 1 -- start start 2024-08-17T02:45:01.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.724+0000 7f7fae200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f8c003a20 0x7f7fa80ad300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.724+0000 7f7fae200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 0x7f7fa8168d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.724+0000 7f7fae200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c005330 0x7f7fa8149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.724+0000 7f7fae200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa816d210 con 0x7f7f8c004780 2024-08-17T02:45:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.724+0000 7f7fae200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80c1960 con 0x7f7f8c005330 2024-08-17T02:45:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.724+0000 7f7fae200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80ad840 con 0x7f7f8c003a20 2024-08-17T02:45:01.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.725+0000 7f7f9fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 0x7f7fa8168d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:01.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.725+0000 7f7fa7400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f8c003a20 0x7f7fa80ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.725+0000 7f7fa7400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f8c003a20 0x7f7fa80ad300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:60938/0 (socket says 172.21.15.16:60938) 2024-08-17T02:45:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.725+0000 7f7f9fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 0x7f7fa8168d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39010/0 (socket says 172.21.15.16:39010) 2024-08-17T02:45:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.725+0000 7f7fa7400640 1 -- 172.21.15.16:0/2343760000 learned_addr learned my addr 172.21.15.16:0/2343760000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.725+0000 7f7fa7e00640 1 --2- 172.21.15.16:0/2343760000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c005330 0x7f7fa8149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.725+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2343760000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f8c003a20 msgr2=0x7f7fa80ad300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.725+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2343760000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f8c003a20 0x7f7fa80ad300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.725+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2343760000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c005330 msgr2=0x7f7fa8149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.725+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2343760000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c005330 0x7f7fa8149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.725+0000 7f7f9fe00640 1 -- 172.21.15.16:0/2343760000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa80c0d50 con 0x7f7f8c004780 2024-08-17T02:45:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.725+0000 7f7f9fe00640 1 --2- 172.21.15.16:0/2343760000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 0x7f7fa8168d40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c230 tx=0x7f7f90038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:01.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.726+0000 7f7f9ea00640 1 -- 172.21.15.16:0/2343760000 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90038880 con 0x7f7f8c004780 2024-08-17T02:45:01.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.726+0000 7f7f9ea00640 1 -- 172.21.15.16:0/2343760000 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f90038a00 con 0x7f7f8c004780 2024-08-17T02:45:01.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.726+0000 7f7fae200640 1 -- 172.21.15.16:0/2343760000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa80c1030 con 0x7f7f8c004780 2024-08-17T02:45:01.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.726+0000 7f7f9ea00640 1 -- 172.21.15.16:0/2343760000 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90038dc0 con 0x7f7f8c004780 2024-08-17T02:45:01.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.726+0000 7f7fae200640 1 -- 172.21.15.16:0/2343760000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa80c15f0 con 0x7f7f8c004780 2024-08-17T02:45:01.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.726+0000 7f7f9ea00640 1 -- 172.21.15.16:0/2343760000 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f7f90041650 con 0x7f7f8c004780 2024-08-17T02:45:01.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.726+0000 7f7f9ea00640 1 -- 172.21.15.16:0/2343760000 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7f9004b740 con 0x7f7f8c004780 2024-08-17T02:45:01.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.727+0000 7f7fae200640 1 -- 172.21.15.16:0/2343760000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa8071050 con 0x7f7f8c004780 2024-08-17T02:45:01.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.731+0000 7f7f9ea00640 1 -- 172.21.15.16:0/2343760000 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f7f900617a0 con 0x7f7f8c004780 2024-08-17T02:45:01.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.803+0000 7f7fae200640 1 -- 172.21.15.16:0/2343760000 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7fa80c1aa0 con 0x7f7f8c004780 2024-08-17T02:45:01.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.806+0000 7f7f9ea00640 1 -- 172.21.15.16:0/2343760000 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7f90050020 con 0x7f7f8c004780 2024-08-17T02:45:01.806 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-08-17T02:45:01.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.808+0000 7f7fae200640 1 -- 172.21.15.16:0/2343760000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 msgr2=0x7f7fa8168d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:01.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.808+0000 7f7fae200640 1 --2- 172.21.15.16:0/2343760000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 0x7f7fa8168d40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c230 tx=0x7f7f90038000 comp rx=0 tx=0).stop 2024-08-17T02:45:01.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.808+0000 7f7fae200640 1 -- 172.21.15.16:0/2343760000 shutdown_connections 2024-08-17T02:45:01.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.808+0000 7f7fae200640 1 --2- 172.21.15.16:0/2343760000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f8c005330 0x7f7fa8149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:01.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.808+0000 7f7fae200640 1 --2- 172.21.15.16:0/2343760000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f8c004780 0x7f7fa8168d40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:01.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.808+0000 7f7fae200640 1 --2- 172.21.15.16:0/2343760000 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7f8c003a20 0x7f7fa80ad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:01.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.808+0000 7f7fae200640 1 -- 172.21.15.16:0/2343760000 >> 172.21.15.16:0/2343760000 conn(0x7f7fa8067750 msgr2=0x7f7fa80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:01.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.808+0000 7f7fae200640 1 -- 172.21.15.16:0/2343760000 shutdown_connections 2024-08-17T02:45:01.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:01.808+0000 7f7fae200640 1 -- 172.21.15.16:0/2343760000 wait complete. 2024-08-17T02:45:01.824 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-08-17T02:45:01.824 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:45:01.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-08-17T02:45:01.853 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:01.853+0000 7fc2eb317ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-17T02:45:01.857 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:01.857+0000 7ffba2166ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-17T02:45:01.882 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:45:01.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-08-17T02:45:01.978 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:45:01.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-08-17T02:45:02.000 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:01.999+0000 7fc2eb317ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-17T02:45:02.013 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:02.013+0000 7ffba2166ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-17T02:45:02.057 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:02.056+0000 7fc2eb317ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-17T02:45:02.069 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-08-17T02:45:02.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-08-17T02:45:02.076 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:02.076+0000 7ffba2166ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-17T02:45:02.131 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:45:02.132 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-08-17T02:45:02.189 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:45:02.189 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-08-17T02:45:02.286 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:45:02.287 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-08-17T02:45:02.362 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:02.362+0000 7ffba2166ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-17T02:45:02.368 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:02.368+0000 7fc2eb317ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-17T02:45:02.383 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-17T02:45:02.383 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-08-17T02:45:02.415 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:02.415+0000 7ffba2166ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-17T02:45:02.421 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:02.420+0000 7fc2eb317ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-17T02:45:02.479 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 6812bdf0-dd96-47a4-aa32-d93f99459266 0 2024-08-17T02:45:02.665 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.665+0000 7f6fcf200640 1 Processor -- start 2024-08-17T02:45:02.666 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.665+0000 7f6fcf200640 1 -- start start 2024-08-17T02:45:02.667 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.666+0000 7f6fcf200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6fd0159b90 0x7f6fd00ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:02.667 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.666+0000 7f6fcf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd00ca8a0 0x7f6fd00cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:02.667 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.666+0000 7f6fcf200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fd0159f60 0x7f6fd00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:02.667 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.666+0000 7f6fcf200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd00cb880 con 0x7f6fd0159f60 2024-08-17T02:45:02.667 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.666+0000 7f6fcf200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd00cd6e0 con 0x7f6fd0159b90 2024-08-17T02:45:02.667 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.666+0000 7f6fcf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00cadc0 con 0x7f6fd00ca8a0 2024-08-17T02:45:02.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.666+0000 7f6fcde00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6fd0159b90 0x7f6fd00ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:02.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.666+0000 7f6fcd400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd00ca8a0 0x7f6fd00cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:02.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.666+0000 7f6fcde00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6fd0159b90 0x7f6fd00ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35582/0 (socket says 172.21.15.142:35582) 2024-08-17T02:45:02.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.666+0000 7f6fcde00640 1 -- 172.21.15.142:0/502605581 learned_addr learned my addr 172.21.15.142:0/502605581 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:02.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.666+0000 7f6fce800640 1 --2- 172.21.15.142:0/502605581 >> v2:172.21.15.16:3300/0 conn(0x7f6fd0159f60 0x7f6fd00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:02.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.667+0000 7f6fcd400640 1 -- 172.21.15.142:0/502605581 >> v2:172.21.15.142:3300/0 conn(0x7f6fd0159b90 msgr2=0x7f6fd00ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:02.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.667+0000 7f6fcd400640 1 --2- 172.21.15.142:0/502605581 >> v2:172.21.15.142:3300/0 conn(0x7f6fd0159b90 0x7f6fd00ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.667+0000 7f6fcd400640 1 -- 172.21.15.142:0/502605581 >> v2:172.21.15.16:3300/0 conn(0x7f6fd0159f60 msgr2=0x7f6fd00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:02.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.667+0000 7f6fcd400640 1 --2- 172.21.15.142:0/502605581 >> v2:172.21.15.16:3300/0 conn(0x7f6fd0159f60 0x7f6fd00ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.667+0000 7f6fcd400640 1 -- 172.21.15.142:0/502605581 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00cb060 con 0x7f6fd00ca8a0 2024-08-17T02:45:02.668 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.667+0000 7f6fcde00640 1 --2- 172.21.15.142:0/502605581 >> v2:172.21.15.142:3300/0 conn(0x7f6fd0159b90 0x7f6fd00ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:02.669 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.668+0000 7f6fcd400640 1 --2- 172.21.15.142:0/502605581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd00ca8a0 0x7f6fd00cc590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6fd008a320 tx=0x7f6fbc0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2ca6b7e23884730 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:02.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.668+0000 7f6fcca00640 1 -- 172.21.15.142:0/502605581 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc02ad00 con 0x7f6fd00ca8a0 2024-08-17T02:45:02.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.668+0000 7f6fcca00640 1 -- 172.21.15.142:0/502605581 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc02ae60 con 0x7f6fd00ca8a0 2024-08-17T02:45:02.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.668+0000 7f6fcca00640 1 -- 172.21.15.142:0/502605581 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc032670 con 0x7f6fd00ca8a0 2024-08-17T02:45:02.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.669+0000 7f6fcf200640 1 -- 172.21.15.142:0/502605581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd00ca8a0 msgr2=0x7f6fd00cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:02.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.669+0000 7f6fcf200640 1 --2- 172.21.15.142:0/502605581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd00ca8a0 0x7f6fd00cc590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6fd008a320 tx=0x7f6fbc0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.669+0000 7f6fcf200640 1 -- 172.21.15.142:0/502605581 shutdown_connections 2024-08-17T02:45:02.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.669+0000 7f6fcf200640 1 --2- 172.21.15.142:0/502605581 >> v2:172.21.15.16:3300/0 conn(0x7f6fd0159f60 0x7f6fd00ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.669+0000 7f6fcf200640 1 --2- 172.21.15.142:0/502605581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd00ca8a0 0x7f6fd00cc590 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.669+0000 7f6fcf200640 1 --2- 172.21.15.142:0/502605581 >> v2:172.21.15.142:3300/0 conn(0x7f6fd0159b90 0x7f6fd00ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.669+0000 7f6fcf200640 1 -- 172.21.15.142:0/502605581 >> 172.21.15.142:0/502605581 conn(0x7f6fd0098e00 msgr2=0x7f6fd006eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:02.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.669+0000 7f6fcf200640 1 -- 172.21.15.142:0/502605581 shutdown_connections 2024-08-17T02:45:02.670 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.669+0000 7f6fcf200640 1 -- 172.21.15.142:0/502605581 wait complete. 2024-08-17T02:45:02.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.670+0000 7f6fcf200640 1 Processor -- start 2024-08-17T02:45:02.671 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.670+0000 7f6fcf200640 1 -- start start 2024-08-17T02:45:02.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.671+0000 7f6fcf200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd00ca8a0 0x7f6fd0169840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:02.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.671+0000 7f6fcf200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fd0159b90 0x7f6fd01549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:02.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.671+0000 7f6fcf200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd0159f60 0x7f6fd00b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:02.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.671+0000 7f6fcf200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0169d80 con 0x7f6fd00ca8a0 2024-08-17T02:45:02.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.671+0000 7f6fcf200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd016c760 con 0x7f6fd0159b90 2024-08-17T02:45:02.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.671+0000 7f6fcf200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd016ca40 con 0x7f6fd0159f60 2024-08-17T02:45:02.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.671+0000 7f6fcd400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fd0159b90 0x7f6fd01549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:02.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.671+0000 7f6fce800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd0159f60 0x7f6fd00b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:02.672 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.672+0000 7f6fcde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd00ca8a0 0x7f6fd0169840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:02.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.672+0000 7f6fce800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd0159f60 0x7f6fd00b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:48686/0 (socket says 172.21.15.142:48686) 2024-08-17T02:45:02.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.672+0000 7f6fce800640 1 -- 172.21.15.142:0/312797492 learned_addr learned my addr 172.21.15.142:0/312797492 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:02.673 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.672+0000 7f6fce800640 1 -- 172.21.15.142:0/312797492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fd0159b90 msgr2=0x7f6fd01549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:02.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.672+0000 7f6fce800640 1 --2- 172.21.15.142:0/312797492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fd0159b90 0x7f6fd01549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.672+0000 7f6fce800640 1 -- 172.21.15.142:0/312797492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd00ca8a0 msgr2=0x7f6fd0169840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:02.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.672+0000 7f6fce800640 1 --2- 172.21.15.142:0/312797492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd00ca8a0 0x7f6fd0169840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.672+0000 7f6fce800640 1 -- 172.21.15.142:0/312797492 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc0009450 con 0x7f6fd0159f60 2024-08-17T02:45:02.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.672+0000 7f6fcd400640 1 --2- 172.21.15.142:0/312797492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fd0159b90 0x7f6fd01549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:02.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.672+0000 7f6fce800640 1 --2- 172.21.15.142:0/312797492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd0159f60 0x7f6fd00b4bb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009e90 tx=0x7f6fc800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:02.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.673+0000 7f6fc5600640 1 -- 172.21.15.142:0/312797492 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800ce10 con 0x7f6fd0159f60 2024-08-17T02:45:02.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.673+0000 7f6fc5600640 1 -- 172.21.15.142:0/312797492 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc800f040 con 0x7f6fd0159f60 2024-08-17T02:45:02.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.673+0000 7f6fc5600640 1 -- 172.21.15.142:0/312797492 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc80146d0 con 0x7f6fd0159f60 2024-08-17T02:45:02.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.673+0000 7f6fcf200640 1 -- 172.21.15.142:0/312797492 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc009390 con 0x7f6fd0159f60 2024-08-17T02:45:02.674 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.673+0000 7f6fcf200640 1 -- 172.21.15.142:0/312797492 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd016d000 con 0x7f6fd0159f60 2024-08-17T02:45:02.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.674+0000 7f6fc5600640 1 -- 172.21.15.142:0/312797492 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f6fc8014d10 con 0x7f6fd0159f60 2024-08-17T02:45:02.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.674+0000 7f6fc5600640 1 -- 172.21.15.142:0/312797492 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6fc80133e0 con 0x7f6fd0159f60 2024-08-17T02:45:02.675 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.674+0000 7f6fcf200640 1 -- 172.21.15.142:0/312797492 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd0071050 con 0x7f6fd0159f60 2024-08-17T02:45:02.677 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.676+0000 7f6fc5600640 1 -- 172.21.15.142:0/312797492 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f6fc8018020 con 0x7f6fd0159f60 2024-08-17T02:45:02.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.750+0000 7f6fcf200640 1 -- 172.21.15.142:0/312797492 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6812bdf0-dd96-47a4-aa32-d93f99459266", "id": 0} v 0) v1 -- 0x7f6fd00c6c50 con 0x7f6fd0159f60 2024-08-17T02:45:02.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.754+0000 7f6fc5600640 1 -- 172.21.15.142:0/312797492 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6812bdf0-dd96-47a4-aa32-d93f99459266", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6fc801d600 con 0x7f6fd0159f60 2024-08-17T02:45:02.754 INFO:teuthology.orchestra.run.smithi142.stdout:0 2024-08-17T02:45:02.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.755+0000 7f6fcf200640 1 -- 172.21.15.142:0/312797492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd0159f60 msgr2=0x7f6fd00b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:02.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.755+0000 7f6fcf200640 1 --2- 172.21.15.142:0/312797492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd0159f60 0x7f6fd00b4bb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8009e90 tx=0x7f6fc800b040 comp rx=0 tx=0).stop 2024-08-17T02:45:02.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.755+0000 7f6fcf200640 1 -- 172.21.15.142:0/312797492 shutdown_connections 2024-08-17T02:45:02.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.755+0000 7f6fcf200640 1 --2- 172.21.15.142:0/312797492 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fd0159f60 0x7f6fd00b4bb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.755+0000 7f6fcf200640 1 --2- 172.21.15.142:0/312797492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fd0159b90 0x7f6fd01549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.755+0000 7f6fcf200640 1 --2- 172.21.15.142:0/312797492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd00ca8a0 0x7f6fd0169840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.755+0000 7f6fcf200640 1 -- 172.21.15.142:0/312797492 >> 172.21.15.142:0/312797492 conn(0x7f6fd0098e00 msgr2=0x7f6fd00673b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:02.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.755+0000 7f6fcf200640 1 -- 172.21.15.142:0/312797492 shutdown_connections 2024-08-17T02:45:02.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.755+0000 7f6fcf200640 1 -- 172.21.15.142:0/312797492 wait complete. 2024-08-17T02:45:02.769 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 591698cf-7733-47dd-9084-c5876b53bbc5 1 2024-08-17T02:45:02.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.913+0000 7fd9b5600640 1 Processor -- start 2024-08-17T02:45:02.914 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.913+0000 7fd9b5600640 1 -- start start 2024-08-17T02:45:02.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9b5600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd9b00aefb0 0x7fd9b00af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:02.915 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9b5600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9b00ae5b0 0x7fd9b00690a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:02.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9b5600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b0068c10 con 0x7fd9b015c7c0 2024-08-17T02:45:02.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9b5600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b00af8c0 con 0x7fd9b00ae5b0 2024-08-17T02:45:02.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9b5600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b0067d60 con 0x7fd9b00aefb0 2024-08-17T02:45:02.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9afe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd9b00aefb0 0x7fd9b00af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:02.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9b4c00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fd9b015c7c0 0x7fd9b015cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:49776/0 (socket says 172.21.15.142:49776) 2024-08-17T02:45:02.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9b4c00640 1 -- 172.21.15.142:0/480178376 learned_addr learned my addr 172.21.15.142:0/480178376 (peer_addr_for_me v1:172.21.15.142:0/0) 2024-08-17T02:45:02.916 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9af400640 1 --2- 172.21.15.142:0/480178376 >> v2:172.21.15.16:3300/0 conn(0x7fd9b00ae5b0 0x7fd9b00690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:02.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9aea00640 1 -- 172.21.15.142:0/480178376 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714005971 0 0) 0x7fd9b0068c10 con 0x7fd9b015c7c0 2024-08-17T02:45:02.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9afe00640 1 -- 172.21.15.142:0/480178376 >> v2:172.21.15.16:3300/0 conn(0x7fd9b00ae5b0 msgr2=0x7fd9b00690a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:02.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9afe00640 1 --2- 172.21.15.142:0/480178376 >> v2:172.21.15.16:3300/0 conn(0x7fd9b00ae5b0 0x7fd9b00690a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9afe00640 1 -- 172.21.15.142:0/480178376 >> v1:172.21.15.142:6789/0 conn(0x7fd9b015c7c0 legacy=0x7fd9b015cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:02.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.915+0000 7fd9afe00640 1 -- 172.21.15.142:0/480178376 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b0068000 con 0x7fd9b00aefb0 2024-08-17T02:45:02.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.916+0000 7fd9afe00640 1 --2- 172.21.15.142:0/480178376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9b00aefb0 0x7fd9b00af380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9b006ebb0 tx=0x7fd9a00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d5f335794f1e7c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:02.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.917+0000 7fd9aea00640 1 -- 172.21.15.142:0/480178376 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a002ad20 con 0x7fd9b00aefb0 2024-08-17T02:45:02.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.917+0000 7fd9aea00640 1 -- 172.21.15.142:0/480178376 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a002ae80 con 0x7fd9b00aefb0 2024-08-17T02:45:02.917 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.917+0000 7fd9aea00640 1 -- 172.21.15.142:0/480178376 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0032670 con 0x7fd9b00aefb0 2024-08-17T02:45:02.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.917+0000 7fd9b5600640 1 -- 172.21.15.142:0/480178376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9b00aefb0 msgr2=0x7fd9b00af380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:02.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.917+0000 7fd9b5600640 1 --2- 172.21.15.142:0/480178376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9b00aefb0 0x7fd9b00af380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd9b006ebb0 tx=0x7fd9a00090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.917+0000 7fd9b5600640 1 -- 172.21.15.142:0/480178376 shutdown_connections 2024-08-17T02:45:02.918 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.917+0000 7fd9b5600640 1 --2- 172.21.15.142:0/480178376 >> v2:172.21.15.16:3300/0 conn(0x7fd9b00ae5b0 0x7fd9b00690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.917+0000 7fd9b5600640 1 --2- 172.21.15.142:0/480178376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9b00aefb0 0x7fd9b00af380 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.917+0000 7fd9b5600640 1 -- 172.21.15.142:0/480178376 >> 172.21.15.142:0/480178376 conn(0x7fd9b0096a70 msgr2=0x7fd9b0096e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:02.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.917+0000 7fd9b5600640 1 -- 172.21.15.142:0/480178376 shutdown_connections 2024-08-17T02:45:02.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.917+0000 7fd9b5600640 1 -- 172.21.15.142:0/480178376 wait complete. 2024-08-17T02:45:02.919 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.919+0000 7fd9b5600640 1 Processor -- start 2024-08-17T02:45:02.920 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.919+0000 7fd9b5600640 1 -- start start 2024-08-17T02:45:02.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9b5600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b00ae5b0 0x7fd9b0155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:02.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9b5600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9b00aefb0 0x7fd9b0145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:02.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9b5600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9b015c7c0 0x7fd9b0097830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:02.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9b5600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b0098060 con 0x7fd9b00ae5b0 2024-08-17T02:45:02.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9b5600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b0098340 con 0x7fd9b00aefb0 2024-08-17T02:45:02.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9b5600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b0098620 con 0x7fd9b015c7c0 2024-08-17T02:45:02.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9b4c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9b015c7c0 0x7fd9b0097830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:02.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9af400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9b00aefb0 0x7fd9b0145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:02.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9b4c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9b015c7c0 0x7fd9b0097830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:48688/0 (socket says 172.21.15.142:48688) 2024-08-17T02:45:02.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9b4c00640 1 -- 172.21.15.142:0/1616974632 learned_addr learned my addr 172.21.15.142:0/1616974632 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:02.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9afe00640 1 --2- 172.21.15.142:0/1616974632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b00ae5b0 0x7fd9b0155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:02.922 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9af400640 1 -- 172.21.15.142:0/1616974632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9b015c7c0 msgr2=0x7fd9b0097830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:02.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9af400640 1 --2- 172.21.15.142:0/1616974632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9b015c7c0 0x7fd9b0097830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9af400640 1 -- 172.21.15.142:0/1616974632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b00ae5b0 msgr2=0x7fd9b0155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:02.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9af400640 1 --2- 172.21.15.142:0/1616974632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b00ae5b0 0x7fd9b0155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:02.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.920+0000 7fd9af400640 1 -- 172.21.15.142:0/1616974632 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a0009390 con 0x7fd9b00aefb0 2024-08-17T02:45:02.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.921+0000 7fd9af400640 1 --2- 172.21.15.142:0/1616974632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9b00aefb0 0x7fd9b0145c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00e900 tx=0x7fd99c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:02.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.921+0000 7fd9a7400640 1 -- 172.21.15.142:0/1616974632 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00ee90 con 0x7fd9b00aefb0 2024-08-17T02:45:02.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.921+0000 7fd9a7400640 1 -- 172.21.15.142:0/1616974632 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c00c940 con 0x7fd9b00aefb0 2024-08-17T02:45:02.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.921+0000 7fd9a7400640 1 -- 172.21.15.142:0/1616974632 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00cd00 con 0x7fd9b00aefb0 2024-08-17T02:45:02.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.921+0000 7fd9b5600640 1 -- 172.21.15.142:0/1616974632 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b0098900 con 0x7fd9b00aefb0 2024-08-17T02:45:02.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.921+0000 7fd9b5600640 1 -- 172.21.15.142:0/1616974632 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b0146160 con 0x7fd9b00aefb0 2024-08-17T02:45:02.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.921+0000 7fd9a7400640 1 -- 172.21.15.142:0/1616974632 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fd99c013680 con 0x7fd9b00aefb0 2024-08-17T02:45:02.923 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.921+0000 7fd9a7400640 1 -- 172.21.15.142:0/1616974632 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fd99c013cf0 con 0x7fd9b00aefb0 2024-08-17T02:45:02.924 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.922+0000 7fd9b5600640 1 -- 172.21.15.142:0/1616974632 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b0071050 con 0x7fd9b00aefb0 2024-08-17T02:45:02.925 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.924+0000 7fd9a7400640 1 -- 172.21.15.142:0/1616974632 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fd99c03b7a0 con 0x7fd9b00aefb0 2024-08-17T02:45:02.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:02.998+0000 7fd9b5600640 1 -- 172.21.15.142:0/1616974632 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "591698cf-7733-47dd-9084-c5876b53bbc5", "id": 1} v 0) v1 -- 0x7fd9b0069590 con 0x7fd9b00aefb0 2024-08-17T02:45:03.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.002+0000 7fd9a7400640 1 -- 172.21.15.142:0/1616974632 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "591698cf-7733-47dd-9084-c5876b53bbc5", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd99c02a020 con 0x7fd9b00aefb0 2024-08-17T02:45:03.003 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-08-17T02:45:03.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.003+0000 7fd9b5600640 1 -- 172.21.15.142:0/1616974632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9b00aefb0 msgr2=0x7fd9b0145c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.003+0000 7fd9b5600640 1 --2- 172.21.15.142:0/1616974632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9b00aefb0 0x7fd9b0145c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd99c00e900 tx=0x7fd99c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.003+0000 7fd9b5600640 1 -- 172.21.15.142:0/1616974632 shutdown_connections 2024-08-17T02:45:03.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.003+0000 7fd9b5600640 1 --2- 172.21.15.142:0/1616974632 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd9b015c7c0 0x7fd9b0097830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.003+0000 7fd9b5600640 1 --2- 172.21.15.142:0/1616974632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9b00aefb0 0x7fd9b0145c20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.003+0000 7fd9b5600640 1 --2- 172.21.15.142:0/1616974632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9b00ae5b0 0x7fd9b0155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.003+0000 7fd9b5600640 1 -- 172.21.15.142:0/1616974632 >> 172.21.15.142:0/1616974632 conn(0x7fd9b0096a70 msgr2=0x7fd9b00ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:03.004 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.003+0000 7fd9b5600640 1 -- 172.21.15.142:0/1616974632 shutdown_connections 2024-08-17T02:45:03.005 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.003+0000 7fd9b5600640 1 -- 172.21.15.142:0/1616974632 wait complete. 2024-08-17T02:45:03.018 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new b8f63e22-f125-40d2-b4bd-065a24d2a7e5 2 2024-08-17T02:45:03.175 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.173+0000 7fe3ff600640 1 Processor -- start 2024-08-17T02:45:03.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.173+0000 7fe3ff600640 1 -- start start 2024-08-17T02:45:03.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.174+0000 7fe3ff600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe3f8155980 0x7fe3f8156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.174+0000 7fe3ff600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80a54d0 0x7fe3f8155d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.174+0000 7fe3ff600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe3f80af580 0x7fe3f80afa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.174+0000 7fe3ff600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80aff40 con 0x7fe3f8155980 2024-08-17T02:45:03.176 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.174+0000 7fe3ff600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f8158400 con 0x7fe3f80af580 2024-08-17T02:45:03.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.174+0000 7fe3ff600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f81577f0 con 0x7fe3f80a54d0 2024-08-17T02:45:03.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.174+0000 7fe3fd600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe3f80af580 0x7fe3f80afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.174+0000 7fe3f7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80a54d0 0x7fe3f8155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.174+0000 7fe3fd600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe3f80af580 0x7fe3f80afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35618/0 (socket says 172.21.15.142:35618) 2024-08-17T02:45:03.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.174+0000 7fe3fd600640 1 -- 172.21.15.142:0/4286834454 learned_addr learned my addr 172.21.15.142:0/4286834454 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:03.177 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.175+0000 7fe3fcc00640 1 --2- 172.21.15.142:0/4286834454 >> v2:172.21.15.16:3300/0 conn(0x7fe3f8155980 0x7fe3f8156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.175+0000 7fe3fd600640 1 -- 172.21.15.142:0/4286834454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80a54d0 msgr2=0x7fe3f8155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:03.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.175+0000 7fe3fd600640 1 --2- 172.21.15.142:0/4286834454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80a54d0 0x7fe3f8155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.175+0000 7fe3fd600640 1 -- 172.21.15.142:0/4286834454 >> v2:172.21.15.16:3300/0 conn(0x7fe3f8155980 msgr2=0x7fe3f8156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:03.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.175+0000 7fe3fd600640 1 --2- 172.21.15.142:0/4286834454 >> v2:172.21.15.16:3300/0 conn(0x7fe3f8155980 0x7fe3f8156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.175+0000 7fe3fd600640 1 -- 172.21.15.142:0/4286834454 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8157ad0 con 0x7fe3f80af580 2024-08-17T02:45:03.178 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.175+0000 7fe3f7e00640 1 --2- 172.21.15.142:0/4286834454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80a54d0 0x7fe3f8155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:03.181 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.176+0000 7fe3fd600640 1 --2- 172.21.15.142:0/4286834454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80af580 0x7fe3f80afa00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe3f806c550 tx=0x7fe3f0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2ab1331107012c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:03.181 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.176+0000 7fe3f7400640 1 -- 172.21.15.142:0/4286834454 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0031400 con 0x7fe3f80af580 2024-08-17T02:45:03.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.176+0000 7fe3f7400640 1 -- 172.21.15.142:0/4286834454 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f002d050 con 0x7fe3f80af580 2024-08-17T02:45:03.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.176+0000 7fe3f7400640 1 -- 172.21.15.142:0/4286834454 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0031790 con 0x7fe3f80af580 2024-08-17T02:45:03.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.177+0000 7fe3ff600640 1 -- 172.21.15.142:0/4286834454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80af580 msgr2=0x7fe3f80afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.177+0000 7fe3ff600640 1 --2- 172.21.15.142:0/4286834454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80af580 0x7fe3f80afa00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe3f806c550 tx=0x7fe3f0009840 comp rx=0 tx=0).stop 2024-08-17T02:45:03.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.177+0000 7fe3ff600640 1 -- 172.21.15.142:0/4286834454 shutdown_connections 2024-08-17T02:45:03.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.177+0000 7fe3ff600640 1 --2- 172.21.15.142:0/4286834454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80af580 0x7fe3f80afa00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.177+0000 7fe3ff600640 1 --2- 172.21.15.142:0/4286834454 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80a54d0 0x7fe3f8155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.177+0000 7fe3ff600640 1 --2- 172.21.15.142:0/4286834454 >> v2:172.21.15.16:3300/0 conn(0x7fe3f8155980 0x7fe3f8156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.182 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.177+0000 7fe3ff600640 1 -- 172.21.15.142:0/4286834454 >> 172.21.15.142:0/4286834454 conn(0x7fe3f8067750 msgr2=0x7fe3f80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:03.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.177+0000 7fe3ff600640 1 -- 172.21.15.142:0/4286834454 shutdown_connections 2024-08-17T02:45:03.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.177+0000 7fe3ff600640 1 -- 172.21.15.142:0/4286834454 wait complete. 2024-08-17T02:45:03.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.178+0000 7fe3ff600640 1 Processor -- start 2024-08-17T02:45:03.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.178+0000 7fe3ff600640 1 -- start start 2024-08-17T02:45:03.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.179+0000 7fe3ff600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80a54d0 0x7fe3f815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.179+0000 7fe3ff600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80af580 0x7fe3f814c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.179+0000 7fe3ff600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f8155980 0x7fe3f814c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.179+0000 7fe3ff600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f8137270 con 0x7fe3f8155980 2024-08-17T02:45:03.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.179+0000 7fe3ff600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f815d4d0 con 0x7fe3f80af580 2024-08-17T02:45:03.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.179+0000 7fe3ff600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f815d7b0 con 0x7fe3f80a54d0 2024-08-17T02:45:03.183 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.180+0000 7fe3fcc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80a54d0 0x7fe3f815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.180+0000 7fe3f7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80af580 0x7fe3f814c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.180+0000 7fe3fcc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80a54d0 0x7fe3f815cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:48698/0 (socket says 172.21.15.142:48698) 2024-08-17T02:45:03.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.180+0000 7fe3fcc00640 1 -- 172.21.15.142:0/1598104688 learned_addr learned my addr 172.21.15.142:0/1598104688 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:03.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.180+0000 7fe3fd600640 1 --2- 172.21.15.142:0/1598104688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f8155980 0x7fe3f814c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.180+0000 7fe3f7e00640 1 -- 172.21.15.142:0/1598104688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80a54d0 msgr2=0x7fe3f815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.180+0000 7fe3f7e00640 1 --2- 172.21.15.142:0/1598104688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80a54d0 0x7fe3f815cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.180+0000 7fe3f7e00640 1 -- 172.21.15.142:0/1598104688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f8155980 msgr2=0x7fe3f814c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.180+0000 7fe3f7e00640 1 --2- 172.21.15.142:0/1598104688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f8155980 0x7fe3f814c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.184 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.180+0000 7fe3f7e00640 1 -- 172.21.15.142:0/1598104688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e4009390 con 0x7fe3f80af580 2024-08-17T02:45:03.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.180+0000 7fe3f7e00640 1 --2- 172.21.15.142:0/1598104688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80af580 0x7fe3f814c2d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400e970 tx=0x7fe3e40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:03.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.180+0000 7fe3ebe00640 1 -- 172.21.15.142:0/1598104688 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400c940 con 0x7fe3f80af580 2024-08-17T02:45:03.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.181+0000 7fe3ebe00640 1 -- 172.21.15.142:0/1598104688 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e400caa0 con 0x7fe3f80af580 2024-08-17T02:45:03.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.181+0000 7fe3ebe00640 1 -- 172.21.15.142:0/1598104688 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400ce60 con 0x7fe3f80af580 2024-08-17T02:45:03.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.181+0000 7fe3ff600640 1 -- 172.21.15.142:0/1598104688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f0009ae0 con 0x7fe3f80af580 2024-08-17T02:45:03.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.181+0000 7fe3ff600640 1 -- 172.21.15.142:0/1598104688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f8136f80 con 0x7fe3f80af580 2024-08-17T02:45:03.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.181+0000 7fe3ebe00640 1 -- 172.21.15.142:0/1598104688 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fe3e4005970 con 0x7fe3f80af580 2024-08-17T02:45:03.185 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.181+0000 7fe3ebe00640 1 -- 172.21.15.142:0/1598104688 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fe3e4013d80 con 0x7fe3f80af580 2024-08-17T02:45:03.186 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.182+0000 7fe3ff600640 1 -- 172.21.15.142:0/1598104688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f8071050 con 0x7fe3f80af580 2024-08-17T02:45:03.187 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.184+0000 7fe3ebe00640 1 -- 172.21.15.142:0/1598104688 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fe3e4021060 con 0x7fe3f80af580 2024-08-17T02:45:03.288 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:03.288+0000 7ffba2166ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-17T02:45:03.292 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.291+0000 7fe3ff600640 1 -- 172.21.15.142:0/1598104688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b8f63e22-f125-40d2-b4bd-065a24d2a7e5", "id": 2} v 0) v1 -- 0x7fe3f815d210 con 0x7fe3f80af580 2024-08-17T02:45:03.295 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.295+0000 7fe3ebe00640 1 -- 172.21.15.142:0/1598104688 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b8f63e22-f125-40d2-b4bd-065a24d2a7e5", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe3e4029020 con 0x7fe3f80af580 2024-08-17T02:45:03.295 INFO:teuthology.orchestra.run.smithi142.stdout:2 2024-08-17T02:45:03.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.296+0000 7fe3ff600640 1 -- 172.21.15.142:0/1598104688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80af580 msgr2=0x7fe3f814c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.296+0000 7fe3ff600640 1 --2- 172.21.15.142:0/1598104688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80af580 0x7fe3f814c2d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400e970 tx=0x7fe3e40090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.296+0000 7fe3ff600640 1 -- 172.21.15.142:0/1598104688 shutdown_connections 2024-08-17T02:45:03.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.296+0000 7fe3ff600640 1 --2- 172.21.15.142:0/1598104688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3f8155980 0x7fe3f814c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.296+0000 7fe3ff600640 1 --2- 172.21.15.142:0/1598104688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3f80af580 0x7fe3f814c2d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.296+0000 7fe3ff600640 1 --2- 172.21.15.142:0/1598104688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe3f80a54d0 0x7fe3f815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.296+0000 7fe3ff600640 1 -- 172.21.15.142:0/1598104688 >> 172.21.15.142:0/1598104688 conn(0x7fe3f8067750 msgr2=0x7fe3f8156be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:03.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.296+0000 7fe3ff600640 1 -- 172.21.15.142:0/1598104688 shutdown_connections 2024-08-17T02:45:03.297 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.296+0000 7fe3ff600640 1 -- 172.21.15.142:0/1598104688 wait complete. 2024-08-17T02:45:03.310 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 5f06efd4-549c-4cd6-a464-b380c84840f4 3 2024-08-17T02:45:03.344 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:03.344+0000 7fc2eb317ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-17T02:45:03.392 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:03.392+0000 7ffba2166ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-17T02:45:03.443 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:03.443+0000 7ffba2166ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-17T02:45:03.448 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:03.448+0000 7fc2eb317ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-17T02:45:03.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.457+0000 7f422c200640 1 Processor -- start 2024-08-17T02:45:03.458 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.457+0000 7f422c200640 1 -- start start 2024-08-17T02:45:03.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.459+0000 7f422c200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 0x7f42240ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.459+0000 7f422c200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42240cac90 con 0x7f42240ca830 2024-08-17T02:45:03.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.459+0000 7f422c200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42240ce9c0 con 0x7f4224158bd0 2024-08-17T02:45:03.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.459+0000 7f422c200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42240cf600 con 0x7f4224158fa0 2024-08-17T02:45:03.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.459+0000 7f4228e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 0x7f42240ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.459+0000 7f4228e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 0x7f42240ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:48710/0 (socket says 172.21.15.142:48710) 2024-08-17T02:45:03.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.459+0000 7f4228e00640 1 -- 172.21.15.142:0/3973772325 learned_addr learned my addr 172.21.15.142:0/3973772325 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:03.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.459+0000 7f4229800640 1 --1- 172.21.15.142:0/3973772325 >> v1:172.21.15.142:6789/0 conn(0x7f4224158bd0 0x7f42240d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:49790/0 (socket says 172.21.15.142:49790) 2024-08-17T02:45:03.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.459+0000 7f4228e00640 1 -- 172.21.15.142:0/3973772325 >> v1:172.21.15.142:6789/0 conn(0x7f4224158bd0 legacy=0x7f42240d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.461 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.459+0000 7f4228e00640 1 -- 172.21.15.142:0/3973772325 >> v1:172.21.15.16:6789/0 conn(0x7f42240ca830 legacy=0x7f42240ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.459+0000 7f4223e00640 1 -- 172.21.15.142:0/3973772325 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135284008 0 0) 0x7f42240ce9c0 con 0x7f4224158bd0 2024-08-17T02:45:03.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.459+0000 7f4228e00640 1 -- 172.21.15.142:0/3973772325 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42240cec60 con 0x7f4224158fa0 2024-08-17T02:45:03.462 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.460+0000 7f4228e00640 1 --2- 172.21.15.142:0/3973772325 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 0x7f42240ca2f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4210009940 tx=0x7f42100091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2e248a5a09908bb server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:03.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.461+0000 7f4223e00640 1 -- 172.21.15.142:0/3973772325 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421002ae40 con 0x7f4224158fa0 2024-08-17T02:45:03.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.461+0000 7f4223e00640 1 -- 172.21.15.142:0/3973772325 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4210032440 con 0x7f4224158fa0 2024-08-17T02:45:03.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.461+0000 7f4223e00640 1 -- 172.21.15.142:0/3973772325 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42100327e0 con 0x7f4224158fa0 2024-08-17T02:45:03.463 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.461+0000 7f422c200640 1 -- 172.21.15.142:0/3973772325 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 msgr2=0x7f42240ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.461+0000 7f422c200640 1 --2- 172.21.15.142:0/3973772325 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 0x7f42240ca2f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4210009940 tx=0x7f42100091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.461+0000 7f422c200640 1 -- 172.21.15.142:0/3973772325 shutdown_connections 2024-08-17T02:45:03.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.461+0000 7f422c200640 1 --2- 172.21.15.142:0/3973772325 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 0x7f42240ca2f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.461+0000 7f422c200640 1 -- 172.21.15.142:0/3973772325 >> 172.21.15.142:0/3973772325 conn(0x7f422406eb90 msgr2=0x7f42240cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:03.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.461+0000 7f422c200640 1 -- 172.21.15.142:0/3973772325 shutdown_connections 2024-08-17T02:45:03.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.461+0000 7f422c200640 1 -- 172.21.15.142:0/3973772325 wait complete. 2024-08-17T02:45:03.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.463+0000 7f422c200640 1 Processor -- start 2024-08-17T02:45:03.464 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.463+0000 7f422c200640 1 -- start start 2024-08-17T02:45:03.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422c200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42240ca830 0x7f42240c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422c200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4224158bd0 0x7f4224158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422c200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 0x7f422415ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422c200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422415a680 con 0x7f4224158bd0 2024-08-17T02:45:03.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422c200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42240c7cb0 con 0x7f42240ca830 2024-08-17T02:45:03.466 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422c200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4224159a70 con 0x7f4224158fa0 2024-08-17T02:45:03.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f4229800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42240ca830 0x7f42240c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422a200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 0x7f422415ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f4229800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42240ca830 0x7f42240c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35626/0 (socket says 172.21.15.142:35626) 2024-08-17T02:45:03.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f4229800640 1 -- 172.21.15.142:0/4019329472 learned_addr learned my addr 172.21.15.142:0/4019329472 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:03.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422a200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 0x7f422415ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:48718/0 (socket says 172.21.15.142:48718) 2024-08-17T02:45:03.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f4228e00640 1 --2- 172.21.15.142:0/4019329472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4224158bd0 0x7f4224158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422a200640 1 -- 172.21.15.142:0/4019329472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42240ca830 msgr2=0x7f42240c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422a200640 1 --2- 172.21.15.142:0/4019329472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42240ca830 0x7f42240c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422a200640 1 -- 172.21.15.142:0/4019329472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4224158bd0 msgr2=0x7f4224158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422a200640 1 --2- 172.21.15.142:0/4019329472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4224158bd0 0x7f4224158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f422a200640 1 -- 172.21.15.142:0/4019329472 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4210009450 con 0x7f4224158fa0 2024-08-17T02:45:03.467 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.464+0000 7f4229800640 1 --2- 172.21.15.142:0/4019329472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42240ca830 0x7f42240c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:03.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.465+0000 7f422a200640 1 --2- 172.21.15.142:0/4019329472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 0x7f422415ab10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f421c00c790 tx=0x7f421c001610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:03.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.465+0000 7f4220c00640 1 -- 172.21.15.142:0/4019329472 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c00a6d0 con 0x7f4224158fa0 2024-08-17T02:45:03.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.465+0000 7f4220c00640 1 -- 172.21.15.142:0/4019329472 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c00a830 con 0x7f4224158fa0 2024-08-17T02:45:03.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.465+0000 7f422c200640 1 -- 172.21.15.142:0/4019329472 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4224159d50 con 0x7f4224158fa0 2024-08-17T02:45:03.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.465+0000 7f4220c00640 1 -- 172.21.15.142:0/4019329472 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c00abf0 con 0x7f4224158fa0 2024-08-17T02:45:03.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.465+0000 7f422c200640 1 -- 172.21.15.142:0/4019329472 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422415a310 con 0x7f4224158fa0 2024-08-17T02:45:03.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.465+0000 7f4220c00640 1 -- 172.21.15.142:0/4019329472 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f421c00ad50 con 0x7f4224158fa0 2024-08-17T02:45:03.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.465+0000 7f4220c00640 1 -- 172.21.15.142:0/4019329472 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f421c012db0 con 0x7f4224158fa0 2024-08-17T02:45:03.468 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.466+0000 7f422c200640 1 -- 172.21.15.142:0/4019329472 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422415a7f0 con 0x7f4224158fa0 2024-08-17T02:45:03.469 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.468+0000 7f4220c00640 1 -- 172.21.15.142:0/4019329472 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f42240cac90 con 0x7f4224158fa0 2024-08-17T02:45:03.500 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:03.500+0000 7fc2eb317ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-17T02:45:03.541 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.541+0000 7f422c200640 1 -- 172.21.15.142:0/4019329472 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5f06efd4-549c-4cd6-a464-b380c84840f4", "id": 3} v 0) v1 -- 0x7f4224071050 con 0x7f4224158fa0 2024-08-17T02:45:03.545 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.544+0000 7f4220c00640 1 -- 172.21.15.142:0/4019329472 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5f06efd4-549c-4cd6-a464-b380c84840f4", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f421c0126d0 con 0x7f4224158fa0 2024-08-17T02:45:03.545 INFO:teuthology.orchestra.run.smithi142.stdout:3 2024-08-17T02:45:03.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.545+0000 7f422c200640 1 -- 172.21.15.142:0/4019329472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 msgr2=0x7f422415ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.545+0000 7f422c200640 1 --2- 172.21.15.142:0/4019329472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 0x7f422415ab10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f421c00c790 tx=0x7f421c001610 comp rx=0 tx=0).stop 2024-08-17T02:45:03.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.546+0000 7f422c200640 1 -- 172.21.15.142:0/4019329472 shutdown_connections 2024-08-17T02:45:03.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.546+0000 7f422c200640 1 --2- 172.21.15.142:0/4019329472 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4224158fa0 0x7f422415ab10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.546+0000 7f422c200640 1 --2- 172.21.15.142:0/4019329472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4224158bd0 0x7f4224158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.546+0000 7f422c200640 1 --2- 172.21.15.142:0/4019329472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42240ca830 0x7f42240c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.546+0000 7f422c200640 1 -- 172.21.15.142:0/4019329472 >> 172.21.15.142:0/4019329472 conn(0x7f422406eb90 msgr2=0x7f42240b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:03.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.546+0000 7f422c200640 1 -- 172.21.15.142:0/4019329472 shutdown_connections 2024-08-17T02:45:03.547 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.546+0000 7f422c200640 1 -- 172.21.15.142:0/4019329472 wait complete. 2024-08-17T02:45:03.560 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 5a4cc2a0-711a-4101-bea6-2a313b9bf80d 4 2024-08-17T02:45:03.619 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:03.619+0000 7ffba2166ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-17T02:45:03.674 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:03.674+0000 7fc2eb317ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-17T02:45:03.675 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:03.675+0000 7ffba2166ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-17T02:45:03.734 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:03.733+0000 7fc2eb317ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-17T02:45:03.739 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:03.739+0000 7ffba2166ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-17T02:45:03.746 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.745+0000 7f546b200640 1 Processor -- start 2024-08-17T02:45:03.746 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.745+0000 7f546b200640 1 -- start start 2024-08-17T02:45:03.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f546b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f546c159aa0 0x7f546c0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f546b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159e70 0x7f546c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f546b200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f546c0cad60 con 0x7f546c15a2c0 2024-08-17T02:45:03.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f546b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f546c0c4b90 con 0x7f546c159aa0 2024-08-17T02:45:03.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f546b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546c0ce630 con 0x7f546c159e70 2024-08-17T02:45:03.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f5469400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159e70 0x7f546c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f5469e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f546c159aa0 0x7f546c0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f5469400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159e70 0x7f546c0cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:48724/0 (socket says 172.21.15.142:48724) 2024-08-17T02:45:03.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f5469400640 1 -- 172.21.15.142:0/1337044823 learned_addr learned my addr 172.21.15.142:0/1337044823 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:03.748 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f5469e00640 1 -- 172.21.15.142:0/1337044823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159e70 msgr2=0x7f546c0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:03.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f5469e00640 1 --2- 172.21.15.142:0/1337044823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159e70 0x7f546c0cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f5469e00640 1 -- 172.21.15.142:0/1337044823 >> v1:172.21.15.16:6789/0 conn(0x7f546c15a2c0 legacy=0x7f546c0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.749 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.747+0000 7f5469e00640 1 -- 172.21.15.142:0/1337044823 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c0ce910 con 0x7f546c159aa0 2024-08-17T02:45:03.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.748+0000 7f5469e00640 1 --2- 172.21.15.142:0/1337044823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f546c159aa0 0x7f546c0ce050 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f545c009850 tx=0x7f545c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef3c316ff4b661e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:03.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.749+0000 7f5468a00640 1 -- 172.21.15.142:0/1337044823 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c02ad90 con 0x7f546c159aa0 2024-08-17T02:45:03.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.749+0000 7f5468a00640 1 -- 172.21.15.142:0/1337044823 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545c032440 con 0x7f546c159aa0 2024-08-17T02:45:03.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.749+0000 7f5468a00640 1 -- 172.21.15.142:0/1337044823 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c032820 con 0x7f546c159aa0 2024-08-17T02:45:03.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.749+0000 7f546b200640 1 -- 172.21.15.142:0/1337044823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f546c159aa0 msgr2=0x7f546c0ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.749+0000 7f546b200640 1 --2- 172.21.15.142:0/1337044823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f546c159aa0 0x7f546c0ce050 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f545c009850 tx=0x7f545c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.749+0000 7f546b200640 1 -- 172.21.15.142:0/1337044823 shutdown_connections 2024-08-17T02:45:03.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.749+0000 7f546b200640 1 --2- 172.21.15.142:0/1337044823 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159e70 0x7f546c0cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.750 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.749+0000 7f546b200640 1 --2- 172.21.15.142:0/1337044823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f546c159aa0 0x7f546c0ce050 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.749+0000 7f546b200640 1 -- 172.21.15.142:0/1337044823 >> 172.21.15.142:0/1337044823 conn(0x7f546c099040 msgr2=0x7f546c06eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:03.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.749+0000 7f546b200640 1 -- 172.21.15.142:0/1337044823 shutdown_connections 2024-08-17T02:45:03.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.749+0000 7f546b200640 1 -- 172.21.15.142:0/1337044823 wait complete. 2024-08-17T02:45:03.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.751+0000 7f546b200640 1 Processor -- start 2024-08-17T02:45:03.751 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.751+0000 7f546b200640 1 -- start start 2024-08-17T02:45:03.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f546b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159aa0 0x7f546c151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f546b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c159e70 0x7f546c0c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f546b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f546c15a2c0 0x7f546c0c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f546b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c0c96d0 con 0x7f546c159e70 2024-08-17T02:45:03.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f546b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c151550 con 0x7f546c15a2c0 2024-08-17T02:45:03.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f546b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546c0c8a50 con 0x7f546c159aa0 2024-08-17T02:45:03.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f5469e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159aa0 0x7f546c151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f5469e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159aa0 0x7f546c151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:48732/0 (socket says 172.21.15.142:48732) 2024-08-17T02:45:03.754 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f5469e00640 1 -- 172.21.15.142:0/1489584129 learned_addr learned my addr 172.21.15.142:0/1489584129 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f546a800640 1 --2- 172.21.15.142:0/1489584129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f546c15a2c0 0x7f546c0c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f5469400640 1 --2- 172.21.15.142:0/1489584129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c159e70 0x7f546c0c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f5469e00640 1 -- 172.21.15.142:0/1489584129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f546c15a2c0 msgr2=0x7f546c0c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f5469e00640 1 --2- 172.21.15.142:0/1489584129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f546c15a2c0 0x7f546c0c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f5469e00640 1 -- 172.21.15.142:0/1489584129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c159e70 msgr2=0x7f546c0c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f5469e00640 1 --2- 172.21.15.142:0/1489584129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c159e70 0x7f546c0c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.752+0000 7f5469e00640 1 -- 172.21.15.142:0/1489584129 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c009390 con 0x7f546c159aa0 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.753+0000 7f5469e00640 1 --2- 172.21.15.142:0/1489584129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159aa0 0x7f546c151fc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f545c009360 tx=0x7f545c032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.753+0000 7f5461600640 1 -- 172.21.15.142:0/1489584129 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c000d10 con 0x7f546c159aa0 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.753+0000 7f5461600640 1 -- 172.21.15.142:0/1489584129 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545c031a60 con 0x7f546c159aa0 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.753+0000 7f5461600640 1 -- 172.21.15.142:0/1489584129 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c031e20 con 0x7f546c159aa0 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.753+0000 7f546b200640 1 -- 172.21.15.142:0/1489584129 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546c0c8d30 con 0x7f546c159aa0 2024-08-17T02:45:03.755 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.753+0000 7f546b200640 1 -- 172.21.15.142:0/1489584129 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546c0c92f0 con 0x7f546c159aa0 2024-08-17T02:45:03.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.753+0000 7f5461600640 1 -- 172.21.15.142:0/1489584129 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f545c043030 con 0x7f546c159aa0 2024-08-17T02:45:03.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.753+0000 7f5461600640 1 -- 172.21.15.142:0/1489584129 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f545c041070 con 0x7f546c159aa0 2024-08-17T02:45:03.756 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.754+0000 7f546b200640 1 -- 172.21.15.142:0/1489584129 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546c071050 con 0x7f546c159aa0 2024-08-17T02:45:03.757 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.756+0000 7f5461600640 1 -- 172.21.15.142:0/1489584129 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f545c04b060 con 0x7f546c159aa0 2024-08-17T02:45:03.797 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:03.797+0000 7fc2eb317ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-17T02:45:03.830 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.830+0000 7f546b200640 1 -- 172.21.15.142:0/1489584129 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5a4cc2a0-711a-4101-bea6-2a313b9bf80d", "id": 4} v 0) v1 -- 0x7f546c06f480 con 0x7f546c159aa0 2024-08-17T02:45:03.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.834+0000 7f5461600640 1 -- 172.21.15.142:0/1489584129 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a4cc2a0-711a-4101-bea6-2a313b9bf80d", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f545c037070 con 0x7f546c159aa0 2024-08-17T02:45:03.834 INFO:teuthology.orchestra.run.smithi142.stdout:4 2024-08-17T02:45:03.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.835+0000 7f546b200640 1 -- 172.21.15.142:0/1489584129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159aa0 msgr2=0x7f546c151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:03.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.835+0000 7f546b200640 1 --2- 172.21.15.142:0/1489584129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159aa0 0x7f546c151fc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f545c009360 tx=0x7f545c032000 comp rx=0 tx=0).stop 2024-08-17T02:45:03.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.835+0000 7f546b200640 1 -- 172.21.15.142:0/1489584129 shutdown_connections 2024-08-17T02:45:03.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.835+0000 7f546b200640 1 --2- 172.21.15.142:0/1489584129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f546c15a2c0 0x7f546c0c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.835+0000 7f546b200640 1 --2- 172.21.15.142:0/1489584129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f546c159e70 0x7f546c0c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.835+0000 7f546b200640 1 --2- 172.21.15.142:0/1489584129 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f546c159aa0 0x7f546c151fc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:03.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.835+0000 7f546b200640 1 -- 172.21.15.142:0/1489584129 >> 172.21.15.142:0/1489584129 conn(0x7f546c099040 msgr2=0x7f546c067580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:03.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.835+0000 7f546b200640 1 -- 172.21.15.142:0/1489584129 shutdown_connections 2024-08-17T02:45:03.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.835+0000 7f546b200640 1 -- 172.21.15.142:0/1489584129 wait complete. 2024-08-17T02:45:03.850 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 6235a39a-b1de-4308-8468-0cf430a64f4c 5 2024-08-17T02:45:03.850 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:03.850+0000 7ffba2166ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-17T02:45:03.910 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:03.910+0000 7fc2eb317ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-17T02:45:03.993 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.993+0000 7f3a62200640 1 Processor -- start 2024-08-17T02:45:03.994 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.993+0000 7f3a62200640 1 -- start start 2024-08-17T02:45:03.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.994+0000 7f3a62200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5c0ca2f0 0x7f3a5c0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.994+0000 7f3a62200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3a5c0c7370 0x7f3a5c0c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:03.995 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.994+0000 7f3a62200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5c06f240 con 0x7f3a5c158ea0 2024-08-17T02:45:03.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.994+0000 7f3a62200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a5c0c6f20 con 0x7f3a5c0c7370 2024-08-17T02:45:03.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.994+0000 7f3a62200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c0cac80 con 0x7f3a5c0ca2f0 2024-08-17T02:45:03.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.995+0000 7f3a5be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3a5c0c7370 0x7f3a5c0c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.995+0000 7f3a5aa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5c0ca2f0 0x7f3a5c0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:03.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.995+0000 7f3a5be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3a5c0c7370 0x7f3a5c0c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35662/0 (socket says 172.21.15.142:35662) 2024-08-17T02:45:03.996 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.995+0000 7f3a5be00640 1 -- 172.21.15.142:0/2830025896 learned_addr learned my addr 172.21.15.142:0/2830025896 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:03.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.995+0000 7f3a5be00640 1 -- 172.21.15.142:0/2830025896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5c0ca2f0 msgr2=0x7f3a5c0ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:03.999 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.995+0000 7f3a5be00640 1 --2- 172.21.15.142:0/2830025896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5c0ca2f0 0x7f3a5c0ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.995+0000 7f3a5be00640 1 -- 172.21.15.142:0/2830025896 >> v1:172.21.15.16:6789/0 conn(0x7f3a5c158ea0 legacy=0x7f3a5c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.995+0000 7f3a5be00640 1 -- 172.21.15.142:0/2830025896 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5c0cf410 con 0x7f3a5c0c7370 2024-08-17T02:45:04.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.996+0000 7f3a5be00640 1 --2- 172.21.15.142:0/2830025896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a5c0c7370 0x7f3a5c0c77f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009f20 tx=0x7f3a54009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d432839b2ff5c60 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:04.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.996+0000 7f3a5a000640 1 -- 172.21.15.142:0/2830025896 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54031400 con 0x7f3a5c0c7370 2024-08-17T02:45:04.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.996+0000 7f3a5a000640 1 -- 172.21.15.142:0/2830025896 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5402d050 con 0x7f3a5c0c7370 2024-08-17T02:45:04.000 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.996+0000 7f3a5a000640 1 -- 172.21.15.142:0/2830025896 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a540317d0 con 0x7f3a5c0c7370 2024-08-17T02:45:04.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.997+0000 7f3a62200640 1 -- 172.21.15.142:0/2830025896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a5c0c7370 msgr2=0x7f3a5c0c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.997+0000 7f3a62200640 1 --2- 172.21.15.142:0/2830025896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a5c0c7370 0x7f3a5c0c77f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009f20 tx=0x7f3a54009840 comp rx=0 tx=0).stop 2024-08-17T02:45:04.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.997+0000 7f3a62200640 1 -- 172.21.15.142:0/2830025896 shutdown_connections 2024-08-17T02:45:04.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.997+0000 7f3a62200640 1 --2- 172.21.15.142:0/2830025896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a5c0c7370 0x7f3a5c0c77f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.997+0000 7f3a62200640 1 --2- 172.21.15.142:0/2830025896 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5c0ca2f0 0x7f3a5c0ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.997+0000 7f3a62200640 1 -- 172.21.15.142:0/2830025896 >> 172.21.15.142:0/2830025896 conn(0x7f3a5c06eb90 msgr2=0x7f3a5c0cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:04.001 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.997+0000 7f3a62200640 1 -- 172.21.15.142:0/2830025896 shutdown_connections 2024-08-17T02:45:04.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.997+0000 7f3a62200640 1 -- 172.21.15.142:0/2830025896 wait complete. 2024-08-17T02:45:04.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.999+0000 7f3a62200640 1 Processor -- start 2024-08-17T02:45:04.002 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:03.999+0000 7f3a62200640 1 -- start start 2024-08-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a62200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5c0c7370 0x7f3a5c14a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a62200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a5c0ca2f0 0x7f3a5c16c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a62200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c158ea0 0x7f3a5c16c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a62200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c14fcf0 con 0x7f3a5c158ea0 2024-08-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a62200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c16d320 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a62200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c09a0d0 con 0x7f3a5c0c7370 2024-08-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a5b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5c0c7370 0x7f3a5c14a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a5aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a5c0ca2f0 0x7f3a5c16c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a5b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5c0c7370 0x7f3a5c14a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:48754/0 (socket says 172.21.15.142:48754) 2024-08-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a5aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a5c0ca2f0 0x7f3a5c16c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35674/0 (socket says 172.21.15.142:35674) 2024-08-17T02:45:04.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a5b400640 1 -- 172.21.15.142:0/3603757466 learned_addr learned my addr 172.21.15.142:0/3603757466 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:04.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a5be00640 1 --2- 172.21.15.142:0/3603757466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c158ea0 0x7f3a5c16c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a5aa00640 1 -- 172.21.15.142:0/3603757466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5c0c7370 msgr2=0x7f3a5c14a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a5aa00640 1 --2- 172.21.15.142:0/3603757466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5c0c7370 0x7f3a5c14a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a5aa00640 1 -- 172.21.15.142:0/3603757466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c158ea0 msgr2=0x7f3a5c16c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.008 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a5aa00640 1 --2- 172.21.15.142:0/3603757466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c158ea0 0x7f3a5c16c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a5aa00640 1 -- 172.21.15.142:0/3603757466 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a54009ae0 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.000+0000 7f3a5b400640 1 --2- 172.21.15.142:0/3603757466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5c0c7370 0x7f3a5c14a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:04.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.001+0000 7f3a5aa00640 1 --2- 172.21.15.142:0/3603757466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a5c0ca2f0 0x7f3a5c16c490 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800e990 tx=0x7f3a480090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:04.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.001+0000 7f3a52a00640 1 -- 172.21.15.142:0/3603757466 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4800c940 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.001+0000 7f3a52a00640 1 -- 172.21.15.142:0/3603757466 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4800caa0 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.001+0000 7f3a52a00640 1 -- 172.21.15.142:0/3603757466 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4800ce20 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.001+0000 7f3a62200640 1 -- 172.21.15.142:0/3603757466 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5c14f0e0 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.009 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.001+0000 7f3a62200640 1 -- 172.21.15.142:0/3603757466 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c14f6a0 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.001+0000 7f3a52a00640 1 -- 172.21.15.142:0/3603757466 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f3a48005970 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.001+0000 7f3a52a00640 1 -- 172.21.15.142:0/3603757466 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f3a48013e30 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.002+0000 7f3a62200640 1 -- 172.21.15.142:0/3603757466 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5c14fea0 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.006+0000 7f3a52a00640 1 -- 172.21.15.142:0/3603757466 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f3a5c14fea0 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.080 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.079+0000 7f3a62200640 1 -- 172.21.15.142:0/3603757466 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6235a39a-b1de-4308-8468-0cf430a64f4c", "id": 5} v 0) v1 -- 0x7f3a5c071050 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.083+0000 7f3a52a00640 1 -- 172.21.15.142:0/3603757466 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6235a39a-b1de-4308-8468-0cf430a64f4c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3a48029020 con 0x7f3a5c0ca2f0 2024-08-17T02:45:04.085 INFO:teuthology.orchestra.run.smithi142.stdout:5 2024-08-17T02:45:04.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.084+0000 7f3a62200640 1 -- 172.21.15.142:0/3603757466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a5c0ca2f0 msgr2=0x7f3a5c16c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.084+0000 7f3a62200640 1 --2- 172.21.15.142:0/3603757466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a5c0ca2f0 0x7f3a5c16c490 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800e990 tx=0x7f3a480090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.084+0000 7f3a62200640 1 -- 172.21.15.142:0/3603757466 shutdown_connections 2024-08-17T02:45:04.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.084+0000 7f3a62200640 1 --2- 172.21.15.142:0/3603757466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5c158ea0 0x7f3a5c16c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.084+0000 7f3a62200640 1 --2- 172.21.15.142:0/3603757466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a5c0ca2f0 0x7f3a5c16c490 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.084+0000 7f3a62200640 1 --2- 172.21.15.142:0/3603757466 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a5c0c7370 0x7f3a5c14a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.084+0000 7f3a62200640 1 -- 172.21.15.142:0/3603757466 >> 172.21.15.142:0/3603757466 conn(0x7f3a5c06eb90 msgr2=0x7f3a5c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:04.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.085+0000 7f3a62200640 1 -- 172.21.15.142:0/3603757466 shutdown_connections 2024-08-17T02:45:04.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.085+0000 7f3a62200640 1 -- 172.21.15.142:0/3603757466 wait complete. 2024-08-17T02:45:04.099 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 178547a0-450d-42c8-80b0-6e887a5877aa 6 2024-08-17T02:45:04.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.242+0000 7faf7ae00640 1 Processor -- start 2024-08-17T02:45:04.243 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.242+0000 7faf7ae00640 1 -- start start 2024-08-17T02:45:04.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.243+0000 7faf7ae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf74158ea0 0x7faf740d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.243+0000 7faf7ae00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf740caa00 0x7faf740c7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.243+0000 7faf7ae00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7406f240 con 0x7faf740d09a0 2024-08-17T02:45:04.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.243+0000 7faf7ae00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf740c6f20 con 0x7faf740caa00 2024-08-17T02:45:04.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.243+0000 7faf7ae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf740cf410 con 0x7faf74158ea0 2024-08-17T02:45:04.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.244+0000 7faf73e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf74158ea0 0x7faf740d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.244+0000 7faf73e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf74158ea0 0x7faf740d0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.142:48756/0 (socket says 172.21.15.142:48756) 2024-08-17T02:45:04.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.244+0000 7faf73e00640 1 -- 172.21.15.142:0/1572240062 learned_addr learned my addr 172.21.15.142:0/1572240062 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:04.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.244+0000 7faf78e00640 1 --2- 172.21.15.142:0/1572240062 >> v2:172.21.15.16:3300/0 conn(0x7faf740caa00 0x7faf740c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.244+0000 7faf73e00640 1 -- 172.21.15.142:0/1572240062 >> v2:172.21.15.16:3300/0 conn(0x7faf740caa00 msgr2=0x7faf740c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:04.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.244+0000 7faf73e00640 1 --2- 172.21.15.142:0/1572240062 >> v2:172.21.15.16:3300/0 conn(0x7faf740caa00 0x7faf740c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.245 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.244+0000 7faf73e00640 1 -- 172.21.15.142:0/1572240062 >> v1:172.21.15.142:6789/0 conn(0x7faf740d09a0 legacy=0x7faf740ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.244+0000 7faf73e00640 1 -- 172.21.15.142:0/1572240062 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf740ceb70 con 0x7faf74158ea0 2024-08-17T02:45:04.246 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.244+0000 7faf72a00640 1 -- 172.21.15.142:0/1572240062 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498819185 0 0) 0x7faf7406f240 con 0x7faf740d09a0 2024-08-17T02:45:04.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.245+0000 7faf73e00640 1 --2- 172.21.15.142:0/1572240062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf74158ea0 0x7faf740d0460 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faf64009910 tx=0x7faf640091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c9259c1c6677ac server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:04.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.245+0000 7faf72a00640 1 -- 172.21.15.142:0/1572240062 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6402adb0 con 0x7faf74158ea0 2024-08-17T02:45:04.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.245+0000 7faf72a00640 1 -- 172.21.15.142:0/1572240062 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf64032440 con 0x7faf74158ea0 2024-08-17T02:45:04.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.245+0000 7faf72a00640 1 -- 172.21.15.142:0/1572240062 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf640327e0 con 0x7faf74158ea0 2024-08-17T02:45:04.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.246+0000 7faf7ae00640 1 -- 172.21.15.142:0/1572240062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf74158ea0 msgr2=0x7faf740d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.246+0000 7faf7ae00640 1 --2- 172.21.15.142:0/1572240062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf74158ea0 0x7faf740d0460 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faf64009910 tx=0x7faf640091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.246+0000 7faf7ae00640 1 -- 172.21.15.142:0/1572240062 shutdown_connections 2024-08-17T02:45:04.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.246+0000 7faf7ae00640 1 --2- 172.21.15.142:0/1572240062 >> v2:172.21.15.16:3300/0 conn(0x7faf740caa00 0x7faf740c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.247 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.246+0000 7faf7ae00640 1 --2- 172.21.15.142:0/1572240062 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf74158ea0 0x7faf740d0460 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.246+0000 7faf7ae00640 1 -- 172.21.15.142:0/1572240062 >> 172.21.15.142:0/1572240062 conn(0x7faf7406eb90 msgr2=0x7faf740cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:04.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.246+0000 7faf7ae00640 1 -- 172.21.15.142:0/1572240062 shutdown_connections 2024-08-17T02:45:04.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.246+0000 7faf7ae00640 1 -- 172.21.15.142:0/1572240062 wait complete. 2024-08-17T02:45:04.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.248+0000 7faf7ae00640 1 Processor -- start 2024-08-17T02:45:04.248 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.248+0000 7faf7ae00640 1 -- start start 2024-08-17T02:45:04.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.248+0000 7faf7ae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf740caa00 0x7faf740b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf7ae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf740d09a0 0x7faf74156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf7ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf74158ea0 0x7faf74099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf7ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740b4c70 con 0x7faf74158ea0 2024-08-17T02:45:04.250 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf7ae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740b4f50 con 0x7faf740caa00 2024-08-17T02:45:04.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf7ae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf740b5230 con 0x7faf740d09a0 2024-08-17T02:45:04.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf73e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf740caa00 0x7faf740b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf73e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf740caa00 0x7faf740b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35686/0 (socket says 172.21.15.142:35686) 2024-08-17T02:45:04.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf73e00640 1 -- 172.21.15.142:0/1306015008 learned_addr learned my addr 172.21.15.142:0/1306015008 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:04.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf73400640 1 --2- 172.21.15.142:0/1306015008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf740d09a0 0x7faf74156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf78e00640 1 --2- 172.21.15.142:0/1306015008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf74158ea0 0x7faf74099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf73e00640 1 -- 172.21.15.142:0/1306015008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf740d09a0 msgr2=0x7faf74156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf73e00640 1 --2- 172.21.15.142:0/1306015008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf740d09a0 0x7faf74156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf73e00640 1 -- 172.21.15.142:0/1306015008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf74158ea0 msgr2=0x7faf74099910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf73e00640 1 --2- 172.21.15.142:0/1306015008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf74158ea0 0x7faf74099910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.249+0000 7faf73e00640 1 -- 172.21.15.142:0/1306015008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf64009450 con 0x7faf740caa00 2024-08-17T02:45:04.251 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.250+0000 7faf73e00640 1 --2- 172.21.15.142:0/1306015008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf740caa00 0x7faf740b4660 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faf64009910 tx=0x7faf64032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:04.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.250+0000 7faf6b400640 1 -- 172.21.15.142:0/1306015008 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64031de0 con 0x7faf740caa00 2024-08-17T02:45:04.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.250+0000 7faf6b400640 1 -- 172.21.15.142:0/1306015008 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf64000c40 con 0x7faf740caa00 2024-08-17T02:45:04.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.250+0000 7faf6b400640 1 -- 172.21.15.142:0/1306015008 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6403d670 con 0x7faf740caa00 2024-08-17T02:45:04.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.250+0000 7faf7ae00640 1 -- 172.21.15.142:0/1306015008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf740b5510 con 0x7faf740caa00 2024-08-17T02:45:04.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.250+0000 7faf7ae00640 1 -- 172.21.15.142:0/1306015008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf741577a0 con 0x7faf740caa00 2024-08-17T02:45:04.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.250+0000 7faf6b400640 1 -- 172.21.15.142:0/1306015008 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7faf64039040 con 0x7faf740caa00 2024-08-17T02:45:04.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.250+0000 7faf6b400640 1 -- 172.21.15.142:0/1306015008 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7faf6403dcf0 con 0x7faf740caa00 2024-08-17T02:45:04.252 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.251+0000 7faf7ae00640 1 -- 172.21.15.142:0/1306015008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7415eb70 con 0x7faf740caa00 2024-08-17T02:45:04.255 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.255+0000 7faf6b400640 1 -- 172.21.15.142:0/1306015008 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7faf64042070 con 0x7faf740caa00 2024-08-17T02:45:04.321 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:04.321+0000 7ffba2166ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-17T02:45:04.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.327+0000 7faf7ae00640 1 -- 172.21.15.142:0/1306015008 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "178547a0-450d-42c8-80b0-6e887a5877aa", "id": 6} v 0) v1 -- 0x7faf7415c690 con 0x7faf740caa00 2024-08-17T02:45:04.332 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.332+0000 7faf6b400640 1 -- 172.21.15.142:0/1306015008 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "178547a0-450d-42c8-80b0-6e887a5877aa", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7faf64037070 con 0x7faf740caa00 2024-08-17T02:45:04.332 INFO:teuthology.orchestra.run.smithi142.stdout:6 2024-08-17T02:45:04.333 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.333+0000 7faf7ae00640 1 -- 172.21.15.142:0/1306015008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf740caa00 msgr2=0x7faf740b4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.334 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.333+0000 7faf7ae00640 1 --2- 172.21.15.142:0/1306015008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf740caa00 0x7faf740b4660 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faf64009910 tx=0x7faf64032000 comp rx=0 tx=0).stop 2024-08-17T02:45:04.334 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.333+0000 7faf7ae00640 1 -- 172.21.15.142:0/1306015008 shutdown_connections 2024-08-17T02:45:04.334 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.333+0000 7faf7ae00640 1 --2- 172.21.15.142:0/1306015008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf74158ea0 0x7faf74099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.334 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.333+0000 7faf7ae00640 1 --2- 172.21.15.142:0/1306015008 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faf740d09a0 0x7faf74156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.334 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.333+0000 7faf7ae00640 1 --2- 172.21.15.142:0/1306015008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf740caa00 0x7faf740b4660 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.334 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.333+0000 7faf7ae00640 1 -- 172.21.15.142:0/1306015008 >> 172.21.15.142:0/1306015008 conn(0x7faf7406eb90 msgr2=0x7faf74157ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:04.334 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.333+0000 7faf7ae00640 1 -- 172.21.15.142:0/1306015008 shutdown_connections 2024-08-17T02:45:04.334 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.333+0000 7faf7ae00640 1 -- 172.21.15.142:0/1306015008 wait complete. 2024-08-17T02:45:04.347 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 7e5915cc-7027-4063-ac18-385c4354a014 7 2024-08-17T02:45:04.388 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:04.387+0000 7fc2eb317ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-17T02:45:04.491 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.491+0000 7fc3fcc00640 1 Processor -- start 2024-08-17T02:45:04.492 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.491+0000 7fc3fcc00640 1 -- start start 2024-08-17T02:45:04.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.492+0000 7fc3fcc00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3f80ca770 0x7fc3f80c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.492+0000 7fc3fcc00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f806f240 con 0x7fc3f8158ea0 2024-08-17T02:45:04.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.492+0000 7fc3fcc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f80cf410 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.492+0000 7fc3fcc00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3f80c6f20 con 0x7fc3f80ca770 2024-08-17T02:45:04.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.493+0000 7fc3efe00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fc3f80ca2f0 0x7fc3f80c7370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.142:49804/0 (socket says 172.21.15.142:49804) 2024-08-17T02:45:04.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.493+0000 7fc3efe00640 1 -- 172.21.15.142:0/2766893685 learned_addr learned my addr 172.21.15.142:0/2766893685 (peer_addr_for_me v1:172.21.15.142:0/0) 2024-08-17T02:45:04.493 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.493+0000 7fc3f6800640 1 --2- 172.21.15.142:0/2766893685 >> v2:172.21.15.16:3300/0 conn(0x7fc3f80ca770 0x7fc3f80c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.493+0000 7fc3f6800640 1 -- 172.21.15.142:0/2766893685 >> v1:172.21.15.142:6789/0 conn(0x7fc3f80ca2f0 legacy=0x7fc3f80c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.493+0000 7fc3f6800640 1 -- 172.21.15.142:0/2766893685 >> v1:172.21.15.16:6789/0 conn(0x7fc3f8158ea0 legacy=0x7fc3f80d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:04.494 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.493+0000 7fc3f6800640 1 -- 172.21.15.142:0/2766893685 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f80d0b70 con 0x7fc3f80ca770 2024-08-17T02:45:04.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.494+0000 7fc3f6800640 1 --2- 172.21.15.142:0/2766893685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80ca770 0x7fc3f80c7a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009e60 tx=0x7fc3f0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fef4d8feff90f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:04.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.495+0000 7fc3f5400640 1 -- 172.21.15.142:0/2766893685 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0007550 con 0x7fc3f80ca770 2024-08-17T02:45:04.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.495+0000 7fc3f5400640 1 -- 172.21.15.142:0/2766893685 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f002d050 con 0x7fc3f80ca770 2024-08-17T02:45:04.495 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.495+0000 7fc3f5400640 1 -- 172.21.15.142:0/2766893685 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0007920 con 0x7fc3f80ca770 2024-08-17T02:45:04.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.495+0000 7fc3fcc00640 1 -- 172.21.15.142:0/2766893685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80ca770 msgr2=0x7fc3f80c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.495+0000 7fc3fcc00640 1 --2- 172.21.15.142:0/2766893685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80ca770 0x7fc3f80c7a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009e60 tx=0x7fc3f0009780 comp rx=0 tx=0).stop 2024-08-17T02:45:04.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.495+0000 7fc3fcc00640 1 -- 172.21.15.142:0/2766893685 shutdown_connections 2024-08-17T02:45:04.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.495+0000 7fc3fcc00640 1 --2- 172.21.15.142:0/2766893685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f80ca770 0x7fc3f80c7a80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.496 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.495+0000 7fc3fcc00640 1 -- 172.21.15.142:0/2766893685 >> 172.21.15.142:0/2766893685 conn(0x7fc3f806eb90 msgr2=0x7fc3f80cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:04.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.495+0000 7fc3fcc00640 1 -- 172.21.15.142:0/2766893685 shutdown_connections 2024-08-17T02:45:04.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.495+0000 7fc3fcc00640 1 -- 172.21.15.142:0/2766893685 wait complete. 2024-08-17T02:45:04.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.497+0000 7fc3fcc00640 1 Processor -- start 2024-08-17T02:45:04.497 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.497+0000 7fc3fcc00640 1 -- start start 2024-08-17T02:45:04.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3fcc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f80ca2f0 0x7fc3f8169b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3fcc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3f80ca770 0x7fc3f80b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3fcc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f8158ea0 0x7fc3f80b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3fcc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f8099c80 con 0x7fc3f8158ea0 2024-08-17T02:45:04.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3fcc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f80b52a0 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3fcc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f80b5520 con 0x7fc3f80ca770 2024-08-17T02:45:04.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3f5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f80ca2f0 0x7fc3f8169b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3efe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3f80ca770 0x7fc3f80b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.500 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3f5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f80ca2f0 0x7fc3f8169b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35700/0 (socket says 172.21.15.142:35700) 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3f5e00640 1 -- 172.21.15.142:0/1673143599 learned_addr learned my addr 172.21.15.142:0/1673143599 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3f6800640 1 --2- 172.21.15.142:0/1673143599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f8158ea0 0x7fc3f80b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3f5e00640 1 -- 172.21.15.142:0/1673143599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3f80ca770 msgr2=0x7fc3f80b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3f5e00640 1 --2- 172.21.15.142:0/1673143599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3f80ca770 0x7fc3f80b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3f5e00640 1 -- 172.21.15.142:0/1673143599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f8158ea0 msgr2=0x7fc3f80b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3f5e00640 1 --2- 172.21.15.142:0/1673143599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f8158ea0 0x7fc3f80b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.498+0000 7fc3f5e00640 1 -- 172.21.15.142:0/1673143599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f0009a20 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.499+0000 7fc3f5e00640 1 --2- 172.21.15.142:0/1673143599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f80ca2f0 0x7fc3f8169b60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800b5e0 tx=0x7fc3e80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.499+0000 7fc3ed600640 1 -- 172.21.15.142:0/1673143599 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800f590 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.499+0000 7fc3ed600640 1 -- 172.21.15.142:0/1673143599 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e800f710 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.499+0000 7fc3fcc00640 1 -- 172.21.15.142:0/1673143599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f80b5800 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.499+0000 7fc3ed600640 1 -- 172.21.15.142:0/1673143599 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800fad0 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.499+0000 7fc3fcc00640 1 -- 172.21.15.142:0/1673143599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f8099910 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.501 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.499+0000 7fc3ed600640 1 -- 172.21.15.142:0/1673143599 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fc3e801a6f0 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.499+0000 7fc3ed600640 1 -- 172.21.15.142:0/1673143599 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fc3e8010a90 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.500+0000 7fc3fcc00640 1 -- 172.21.15.142:0/1673143599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f80b2750 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.502+0000 7fc3ed600640 1 -- 172.21.15.142:0/1673143599 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fc3f80b2750 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.577 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.577+0000 7fc3fcc00640 1 -- 172.21.15.142:0/1673143599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7e5915cc-7027-4063-ac18-385c4354a014", "id": 7} v 0) v1 -- 0x7fc3f8071050 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.585 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.585+0000 7fc3ed600640 1 -- 172.21.15.142:0/1673143599 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7e5915cc-7027-4063-ac18-385c4354a014", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc3e8010d70 con 0x7fc3f80ca2f0 2024-08-17T02:45:04.586 INFO:teuthology.orchestra.run.smithi142.stdout:7 2024-08-17T02:45:04.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.586+0000 7fc3fcc00640 1 -- 172.21.15.142:0/1673143599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f80ca2f0 msgr2=0x7fc3f8169b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.586+0000 7fc3fcc00640 1 --2- 172.21.15.142:0/1673143599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f80ca2f0 0x7fc3f8169b60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800b5e0 tx=0x7fc3e80090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.586+0000 7fc3fcc00640 1 -- 172.21.15.142:0/1673143599 shutdown_connections 2024-08-17T02:45:04.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.586+0000 7fc3fcc00640 1 --2- 172.21.15.142:0/1673143599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f8158ea0 0x7fc3f80b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.587 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.586+0000 7fc3fcc00640 1 --2- 172.21.15.142:0/1673143599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc3f80ca770 0x7fc3f80b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.586+0000 7fc3fcc00640 1 --2- 172.21.15.142:0/1673143599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc3f80ca2f0 0x7fc3f8169b60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.586+0000 7fc3fcc00640 1 -- 172.21.15.142:0/1673143599 >> 172.21.15.142:0/1673143599 conn(0x7fc3f806eb90 msgr2=0x7fc3f80c8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:04.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.586+0000 7fc3fcc00640 1 -- 172.21.15.142:0/1673143599 shutdown_connections 2024-08-17T02:45:04.588 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-17T02:45:04.586+0000 7fc3fcc00640 1 -- 172.21.15.142:0/1673143599 wait complete. 2024-08-17T02:45:04.601 INFO:tasks.ceph.osd.0:Restarting daemon 2024-08-17T02:45:04.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-08-17T02:45:04.602 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:04.602+0000 7ffba2166ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-17T02:45:04.606 INFO:tasks.ceph.osd.0:Started 2024-08-17T02:45:04.606 INFO:tasks.ceph.osd.1:Restarting daemon 2024-08-17T02:45:04.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-08-17T02:45:04.610 INFO:tasks.ceph.osd.1:Started 2024-08-17T02:45:04.610 INFO:tasks.ceph.osd.2:Restarting daemon 2024-08-17T02:45:04.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-08-17T02:45:04.615 INFO:tasks.ceph.osd.2:Started 2024-08-17T02:45:04.615 INFO:tasks.ceph.osd.3:Restarting daemon 2024-08-17T02:45:04.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-08-17T02:45:04.620 INFO:tasks.ceph.osd.3:Started 2024-08-17T02:45:04.620 INFO:tasks.ceph.osd.4:Restarting daemon 2024-08-17T02:45:04.620 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-08-17T02:45:04.646 INFO:tasks.ceph.osd.4:Started 2024-08-17T02:45:04.647 INFO:tasks.ceph.osd.5:Restarting daemon 2024-08-17T02:45:04.647 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-08-17T02:45:04.652 INFO:tasks.ceph.osd.5:Started 2024-08-17T02:45:04.652 INFO:tasks.ceph.osd.6:Restarting daemon 2024-08-17T02:45:04.652 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-08-17T02:45:04.657 INFO:tasks.ceph.osd.6:Started 2024-08-17T02:45:04.658 INFO:tasks.ceph.osd.7:Restarting daemon 2024-08-17T02:45:04.658 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-08-17T02:45:04.663 INFO:tasks.ceph.osd.7:Started 2024-08-17T02:45:04.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:04.677 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:04.676+0000 7fc2eb317ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-17T02:45:04.686 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:04.685+0000 7ffba2166ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-17T02:45:04.750 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:04.750+0000 7fc2eb317ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-17T02:45:04.841 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:04.841+0000 7ffba2166ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-17T02:45:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.883+0000 7f9b03600640 1 Processor -- start 2024-08-17T02:45:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.883+0000 7f9b03600640 1 -- start start 2024-08-17T02:45:04.884 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:04.883+0000 7fc2eb317ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-17T02:45:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.884+0000 7f9b03600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 0x7f9afc0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.884+0000 7f9b03600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9afc06df00 0x7f9afc0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.884+0000 7f9b03600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc156670 con 0x7f9afc155df0 2024-08-17T02:45:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.884+0000 7f9b03600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc0b0350 con 0x7f9afc06df00 2024-08-17T02:45:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.884+0000 7f9b03600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc06e5e0 con 0x7f9afc155a20 2024-08-17T02:45:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.885+0000 7f9b01600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9afc06df00 0x7f9afc0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.885+0000 7f9b01600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9afc06df00 0x7f9afc0ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53510/0 (socket says 172.21.15.16:53510) 2024-08-17T02:45:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.885+0000 7f9b01600640 1 -- 172.21.15.16:0/3058773224 learned_addr learned my addr 172.21.15.16:0/3058773224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.885+0000 7f9b00c00640 1 --2- 172.21.15.16:0/3058773224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 0x7f9afc0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.885+0000 7f9b00c00640 1 -- 172.21.15.16:0/3058773224 >> v2:172.21.15.16:3300/0 conn(0x7f9afc06df00 msgr2=0x7f9afc0ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.885+0000 7f9b00c00640 1 --2- 172.21.15.16:0/3058773224 >> v2:172.21.15.16:3300/0 conn(0x7f9afc06df00 0x7f9afc0ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.885+0000 7f9afb400640 1 -- 172.21.15.16:0/3058773224 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654547028 0 0) 0x7f9afc156670 con 0x7f9afc155df0 2024-08-17T02:45:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.885+0000 7f9b00c00640 1 -- 172.21.15.16:0/3058773224 >> v1:172.21.15.142:6789/0 conn(0x7f9afc155df0 legacy=0x7f9afc06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.885+0000 7f9b00c00640 1 -- 172.21.15.16:0/3058773224 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc06e8c0 con 0x7f9afc155a20 2024-08-17T02:45:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.885+0000 7f9b01600640 1 --2- 172.21.15.16:0/3058773224 >> v2:172.21.15.16:3300/0 conn(0x7f9afc06df00 0x7f9afc0ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.886+0000 7f9b00c00640 1 --2- 172.21.15.16:0/3058773224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 0x7f9afc0adf10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009850 tx=0x7f9af00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c5196ee6fd41854 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.886+0000 7f9afb400640 1 -- 172.21.15.16:0/3058773224 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af002ad90 con 0x7f9afc155a20 2024-08-17T02:45:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.886+0000 7f9afb400640 1 -- 172.21.15.16:0/3058773224 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af0032440 con 0x7f9afc155a20 2024-08-17T02:45:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.886+0000 7f9afb400640 1 -- 172.21.15.16:0/3058773224 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00327e0 con 0x7f9afc155a20 2024-08-17T02:45:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.887+0000 7f9b03600640 1 -- 172.21.15.16:0/3058773224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 msgr2=0x7f9afc0adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.887+0000 7f9b03600640 1 --2- 172.21.15.16:0/3058773224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 0x7f9afc0adf10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009850 tx=0x7f9af00090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.887+0000 7f9b03600640 1 -- 172.21.15.16:0/3058773224 shutdown_connections 2024-08-17T02:45:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.887+0000 7f9b03600640 1 --2- 172.21.15.16:0/3058773224 >> v2:172.21.15.16:3300/0 conn(0x7f9afc06df00 0x7f9afc0ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.887+0000 7f9b03600640 1 --2- 172.21.15.16:0/3058773224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 0x7f9afc0adf10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.887+0000 7f9b03600640 1 -- 172.21.15.16:0/3058773224 >> 172.21.15.16:0/3058773224 conn(0x7f9afc0cb9b0 msgr2=0x7f9afc0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.887+0000 7f9b03600640 1 -- 172.21.15.16:0/3058773224 shutdown_connections 2024-08-17T02:45:04.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.887+0000 7f9b03600640 1 -- 172.21.15.16:0/3058773224 wait complete. 2024-08-17T02:45:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.889+0000 7f9b03600640 1 Processor -- start 2024-08-17T02:45:04.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.889+0000 7f9b03600640 1 -- start start 2024-08-17T02:45:04.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.890+0000 7f9b03600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9afc06df00 0x7f9afc16a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.890+0000 7f9b03600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 0x7f9afc1670d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.890+0000 7f9b03600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155df0 0x7f9afc16bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.890+0000 7f9b03600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc16aba0 con 0x7f9afc155df0 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.890+0000 7f9b03600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc16ae70 con 0x7f9afc06df00 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.890+0000 7f9b03600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc16b150 con 0x7f9afc155a20 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.890+0000 7f9b01600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155df0 0x7f9afc16bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.890+0000 7f9afbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 0x7f9afc1670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.890+0000 7f9b00c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9afc06df00 0x7f9afc16a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.890+0000 7f9afbe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 0x7f9afc1670d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:41494/0 (socket says 172.21.15.16:41494) 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.890+0000 7f9afbe00640 1 -- 172.21.15.16:0/901168445 learned_addr learned my addr 172.21.15.16:0/901168445 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.891+0000 7f9afbe00640 1 -- 172.21.15.16:0/901168445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9afc06df00 msgr2=0x7f9afc16a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.891+0000 7f9afbe00640 1 --2- 172.21.15.16:0/901168445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9afc06df00 0x7f9afc16a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.891+0000 7f9afbe00640 1 -- 172.21.15.16:0/901168445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155df0 msgr2=0x7f9afc16bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.891+0000 7f9afbe00640 1 --2- 172.21.15.16:0/901168445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155df0 0x7f9afc16bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.891+0000 7f9afbe00640 1 -- 172.21.15.16:0/901168445 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af4009ae0 con 0x7f9afc155a20 2024-08-17T02:45:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.891+0000 7f9afbe00640 1 --2- 172.21.15.16:0/901168445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 0x7f9afc1670d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800c4e0 tx=0x7f9ae8007ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.891+0000 7f9aefe00640 1 -- 172.21.15.16:0/901168445 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8011590 con 0x7f9afc155a20 2024-08-17T02:45:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.891+0000 7f9aefe00640 1 -- 172.21.15.16:0/901168445 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae80116f0 con 0x7f9afc155a20 2024-08-17T02:45:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.891+0000 7f9b03600640 1 -- 172.21.15.16:0/901168445 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af0009390 con 0x7f9afc155a20 2024-08-17T02:45:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.891+0000 7f9aefe00640 1 -- 172.21.15.16:0/901168445 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8011af0 con 0x7f9afc155a20 2024-08-17T02:45:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.891+0000 7f9b03600640 1 -- 172.21.15.16:0/901168445 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc16b710 con 0x7f9afc155a20 2024-08-17T02:45:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.891+0000 7f9aefe00640 1 -- 172.21.15.16:0/901168445 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f9ae801c440 con 0x7f9afc155a20 2024-08-17T02:45:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.892+0000 7f9aefe00640 1 -- 172.21.15.16:0/901168445 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f9ae8012b00 con 0x7f9afc155a20 2024-08-17T02:45:04.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.892+0000 7f9b03600640 1 -- 172.21.15.16:0/901168445 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc064db0 con 0x7f9afc155a20 2024-08-17T02:45:04.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.895+0000 7f9aefe00640 1 -- 172.21.15.16:0/901168445 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f9ae8017070 con 0x7f9afc155a20 2024-08-17T02:45:04.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.958+0000 7f9b03600640 1 -- 172.21.15.16:0/901168445 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9afc0af210 con 0x7f9afc155a20 2024-08-17T02:45:04.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.959+0000 7f9aefe00640 1 -- 172.21.15.16:0/901168445 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f9ae801c6f0 con 0x7f9afc155a20 2024-08-17T02:45:04.960 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:04.960 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:04.578707+0000","last_up_change":"0.000000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.960+0000 7f9b03600640 1 -- 172.21.15.16:0/901168445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 msgr2=0x7f9afc1670d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.960+0000 7f9b03600640 1 --2- 172.21.15.16:0/901168445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 0x7f9afc1670d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800c4e0 tx=0x7f9ae8007ef0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.960+0000 7f9b03600640 1 -- 172.21.15.16:0/901168445 shutdown_connections 2024-08-17T02:45:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.960+0000 7f9b03600640 1 --2- 172.21.15.16:0/901168445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9afc155df0 0x7f9afc16bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.960+0000 7f9b03600640 1 --2- 172.21.15.16:0/901168445 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9afc155a20 0x7f9afc1670d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.960+0000 7f9b03600640 1 --2- 172.21.15.16:0/901168445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9afc06df00 0x7f9afc16a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.960+0000 7f9b03600640 1 -- 172.21.15.16:0/901168445 >> 172.21.15.16:0/901168445 conn(0x7f9afc0cb9b0 msgr2=0x7f9afc069ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.960+0000 7f9b03600640 1 -- 172.21.15.16:0/901168445 shutdown_connections 2024-08-17T02:45:04.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:04.960+0000 7f9b03600640 1 -- 172.21.15.16:0/901168445 wait complete. 2024-08-17T02:45:04.975 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-08-17T02:45:04.975 INFO:tasks.ceph:Waiting for OSDs to come up 2024-08-17T02:45:05.141 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:05.141+0000 7ffba2166ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-17T02:45:05.172 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:05.171+0000 7fc2eb317ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-17T02:45:05.197 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:05.197+0000 7ffba2166ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-17T02:45:05.224 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:05.224+0000 7fc2eb317ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-17T02:45:05.306 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-08-17T02:45:05.305+0000 7fc13b2752c0 -1 Falling back to public interface 2024-08-17T02:45:05.306 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-08-17T02:45:05.306+0000 7feba6c0c2c0 -1 Falling back to public interface 2024-08-17T02:45:05.312 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-08-17T02:45:05.312+0000 7f95ff20c2c0 -1 Falling back to public interface 2024-08-17T02:45:05.314 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-08-17T02:45:05.314+0000 7f381017b2c0 -1 Falling back to public interface 2024-08-17T02:45:05.322 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-08-17T02:45:05.322+0000 7f325b2122c0 -1 Falling back to public interface 2024-08-17T02:45:05.330 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-08-17T02:45:05.330+0000 7fbbe0e102c0 -1 Falling back to public interface 2024-08-17T02:45:05.345 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-08-17T02:45:05.345+0000 7fa0e64392c0 -1 Falling back to public interface 2024-08-17T02:45:05.346 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-08-17T02:45:05.346+0000 7f397451f2c0 -1 Falling back to public interface 2024-08-17T02:45:05.349 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-08-17T02:45:05.348+0000 7fa0e4800640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6821/0: (98) Address already in use 2024-08-17T02:45:05.392 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:05.392+0000 7ffba2166ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-17T02:45:05.403 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:05.403+0000 7fc2eb317ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-17T02:45:05.451 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:05.451+0000 7ffba2166ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-17T02:45:05.459 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:05.459+0000 7fc2eb317ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-17T02:45:05.533 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:05.533+0000 7ffba2166ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-17T02:45:05.536 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:05.536+0000 7fc2eb317ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-17T02:45:05.587 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:05.587+0000 7ffba2166ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-17T02:45:05.587 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:05.587+0000 7fc2eb317ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-17T02:45:05.783 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:05.917 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.917+0000 7fd1fe800640 1 Processor -- start 2024-08-17T02:45:05.917 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.917+0000 7fd1fe800640 1 -- start start 2024-08-17T02:45:05.918 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.918+0000 7fd1fe800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80b0ac0 0x7fd1f80b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:05.918 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.918+0000 7fd1fe800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1f80b2c60 0x7fd1f80b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:05.919 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.918+0000 7fd1fe800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f80b3620 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.919 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.918+0000 7fd1fe800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f80af240 con 0x7fd1f80b2c60 2024-08-17T02:45:05.919 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.918+0000 7fd1fe800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f80afa20 con 0x7fd1f80b0ac0 2024-08-17T02:45:05.919 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.918+0000 7fd1f7400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd1f80ad8e0 0x7fd1f80d04b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43394/0 (socket says 172.21.15.16:43394) 2024-08-17T02:45:05.919 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.919+0000 7fd1f7400640 1 -- 172.21.15.16:0/1068974881 learned_addr learned my addr 172.21.15.16:0/1068974881 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:05.919 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.918+0000 7fd1f7e00640 1 --2- 172.21.15.16:0/1068974881 >> v2:172.21.15.16:3300/0 conn(0x7fd1f80b2c60 0x7fd1f80b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:05.919 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.919+0000 7fd1f6a00640 1 --2- 172.21.15.16:0/1068974881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80b0ac0 0x7fd1f80b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:05.920 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.919+0000 7fd1f7e00640 1 -- 172.21.15.16:0/1068974881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80b0ac0 msgr2=0x7fd1f80b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:05.920 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.919+0000 7fd1f7e00640 1 --2- 172.21.15.16:0/1068974881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80b0ac0 0x7fd1f80b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:05.920 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.919+0000 7fd1f7e00640 1 -- 172.21.15.16:0/1068974881 >> v1:172.21.15.16:6789/0 conn(0x7fd1f80ad8e0 legacy=0x7fd1f80d04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:05.920 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.919+0000 7fd1f7e00640 1 -- 172.21.15.16:0/1068974881 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f80afd00 con 0x7fd1f80b2c60 2024-08-17T02:45:05.920 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.919+0000 7fd1f6000640 1 -- 172.21.15.16:0/1068974881 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101530782 0 0) 0x7fd1f80b3620 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.920 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.920+0000 7fd1f7e00640 1 --2- 172.21.15.16:0/1068974881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f80b2c60 0x7fd1f80b30e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009e60 tx=0x7fd1f0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d7c40e9bb5e2ec server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:05.921 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.920+0000 7fd1f6000640 1 -- 172.21.15.16:0/1068974881 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0007550 con 0x7fd1f80b2c60 2024-08-17T02:45:05.921 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.920+0000 7fd1f6000640 1 -- 172.21.15.16:0/1068974881 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f002d050 con 0x7fd1f80b2c60 2024-08-17T02:45:05.921 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.920+0000 7fd1f6000640 1 -- 172.21.15.16:0/1068974881 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00078e0 con 0x7fd1f80b2c60 2024-08-17T02:45:05.921 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.921+0000 7fd1fe800640 1 -- 172.21.15.16:0/1068974881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f80b2c60 msgr2=0x7fd1f80b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:05.922 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.921+0000 7fd1fe800640 1 --2- 172.21.15.16:0/1068974881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f80b2c60 0x7fd1f80b30e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009e60 tx=0x7fd1f0009780 comp rx=0 tx=0).stop 2024-08-17T02:45:05.922 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.921+0000 7fd1fe800640 1 -- 172.21.15.16:0/1068974881 shutdown_connections 2024-08-17T02:45:05.922 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.921+0000 7fd1fe800640 1 --2- 172.21.15.16:0/1068974881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f80b2c60 0x7fd1f80b30e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:05.922 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.921+0000 7fd1fe800640 1 --2- 172.21.15.16:0/1068974881 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80b0ac0 0x7fd1f80b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:05.922 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.921+0000 7fd1fe800640 1 -- 172.21.15.16:0/1068974881 >> 172.21.15.16:0/1068974881 conn(0x7fd1f806ec90 msgr2=0x7fd1f80cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:05.922 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.921+0000 7fd1fe800640 1 -- 172.21.15.16:0/1068974881 shutdown_connections 2024-08-17T02:45:05.922 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.921+0000 7fd1fe800640 1 -- 172.21.15.16:0/1068974881 wait complete. 2024-08-17T02:45:05.923 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.922+0000 7fd1fe800640 1 Processor -- start 2024-08-17T02:45:05.923 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.923+0000 7fd1fe800640 1 -- start start 2024-08-17T02:45:05.924 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.923+0000 7fd1fe800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f80ad8e0 0x7fd1f8169650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:05.924 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.923+0000 7fd1fe800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f80b0ac0 0x7fd1f80cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:05.924 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.923+0000 7fd1fe800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80b2c60 0x7fd1f816c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:05.924 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.923+0000 7fd1fe800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f81691c0 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.925 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.923+0000 7fd1fe800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f815dd80 con 0x7fd1f80b0ac0 2024-08-17T02:45:05.925 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.923+0000 7fd1fe800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f816cc70 con 0x7fd1f80b2c60 2024-08-17T02:45:05.925 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.924+0000 7fd1f7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f80ad8e0 0x7fd1f8169650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:05.925 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.924+0000 7fd1f7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f80ad8e0 0x7fd1f8169650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53532/0 (socket says 172.21.15.16:53532) 2024-08-17T02:45:05.925 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.924+0000 7fd1f7400640 1 -- 172.21.15.16:0/1761686243 learned_addr learned my addr 172.21.15.16:0/1761686243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:05.925 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.924+0000 7fd1f6a00640 1 --2- 172.21.15.16:0/1761686243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f80b0ac0 0x7fd1f80cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:05.925 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.924+0000 7fd1f7e00640 1 --2- 172.21.15.16:0/1761686243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80b2c60 0x7fd1f816c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:05.925 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.924+0000 7fd1f7400640 1 -- 172.21.15.16:0/1761686243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80b2c60 msgr2=0x7fd1f816c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:05.925 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.924+0000 7fd1f7400640 1 --2- 172.21.15.16:0/1761686243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80b2c60 0x7fd1f816c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:05.925 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.924+0000 7fd1f7400640 1 -- 172.21.15.16:0/1761686243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f80b0ac0 msgr2=0x7fd1f80cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:05.926 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.924+0000 7fd1f7400640 1 --2- 172.21.15.16:0/1761686243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f80b0ac0 0x7fd1f80cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:05.926 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.924+0000 7fd1f7400640 1 -- 172.21.15.16:0/1761686243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f0009a20 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.926 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.924+0000 7fd1f7e00640 1 --2- 172.21.15.16:0/1761686243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80b2c60 0x7fd1f816c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:05.926 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.924+0000 7fd1f7400640 1 --2- 172.21.15.16:0/1761686243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f80ad8e0 0x7fd1f8169650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800c510 tx=0x7fd1e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:05.927 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.925+0000 7fd1eea00640 1 -- 172.21.15.16:0/1761686243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8005aa0 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.927 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.925+0000 7fd1eea00640 1 -- 172.21.15.16:0/1761686243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e8005c00 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.927 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.925+0000 7fd1fe800640 1 -- 172.21.15.16:0/1761686243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f816cf50 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.927 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.925+0000 7fd1eea00640 1 -- 172.21.15.16:0/1761686243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80070d0 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.927 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.925+0000 7fd1fe800640 1 -- 172.21.15.16:0/1761686243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f8168b00 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.927 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.925+0000 7fd1eea00640 1 -- 172.21.15.16:0/1761686243 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fd1e8007500 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.927 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.925+0000 7fd1eea00640 1 -- 172.21.15.16:0/1761686243 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fd1e801c440 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.927 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.926+0000 7fd1fe800640 1 -- 172.21.15.16:0/1761686243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f8064a50 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.928 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.928+0000 7fd1eea00640 1 -- 172.21.15.16:0/1761686243 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fd1e8017070 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.992 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.992+0000 7fd1fe800640 1 -- 172.21.15.16:0/1761686243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1f80b5c80 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.993 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.993+0000 7fd1eea00640 1 -- 172.21.15.16:0/1761686243 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fd1e8024410 con 0x7fd1f80ad8e0 2024-08-17T02:45:05.993 INFO:teuthology.misc.health.smithi016.stdout: 2024-08-17T02:45:05.993 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":10,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:04.578707+0000","last_up_change":"0.000000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:05.994 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.993+0000 7fd1fe800640 1 -- 172.21.15.16:0/1761686243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f80ad8e0 msgr2=0x7fd1f8169650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:05.994 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.994+0000 7fd1fe800640 1 --2- 172.21.15.16:0/1761686243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f80ad8e0 0x7fd1f8169650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800c510 tx=0x7fd1e800c910 comp rx=0 tx=0).stop 2024-08-17T02:45:05.994 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.994+0000 7fd1fe800640 1 -- 172.21.15.16:0/1761686243 shutdown_connections 2024-08-17T02:45:05.995 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.994+0000 7fd1fe800640 1 --2- 172.21.15.16:0/1761686243 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd1f80b2c60 0x7fd1f816c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:05.995 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.994+0000 7fd1fe800640 1 --2- 172.21.15.16:0/1761686243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd1f80b0ac0 0x7fd1f80cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:05.995 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.994+0000 7fd1fe800640 1 --2- 172.21.15.16:0/1761686243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1f80ad8e0 0x7fd1f8169650 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:05.995 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.994+0000 7fd1fe800640 1 -- 172.21.15.16:0/1761686243 >> 172.21.15.16:0/1761686243 conn(0x7fd1f806ec90 msgr2=0x7fd1f80b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:05.995 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.994+0000 7fd1fe800640 1 -- 172.21.15.16:0/1761686243 shutdown_connections 2024-08-17T02:45:05.995 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:05.994+0000 7fd1fe800640 1 -- 172.21.15.16:0/1761686243 wait complete. 2024-08-17T02:45:06.005 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-08-17T02:45:06.109 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:06.108+0000 7fc2eb317ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-17T02:45:06.144 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:06.144+0000 7ffba2166ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-17T02:45:06.220 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:06.219+0000 7fc2eb317ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-17T02:45:06.266 INFO:tasks.ceph.mgr.y.smithi016.stderr:2024-08-17T02:45:06.266+0000 7ffba2166ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-17T02:45:07.241 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:07.241+0000 7fc2cfe00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-17T02:45:07.242 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-08-17T02:45:07.242+0000 7fc2cfe00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-08-17T02:45:08.836 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-08-17T02:45:08.836+0000 7f95ff20c2c0 -1 osd.6 0 log_to_monitors true 2024-08-17T02:45:08.861 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-08-17T02:45:08.860+0000 7feba6c0c2c0 -1 osd.5 0 log_to_monitors true 2024-08-17T02:45:08.875 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-08-17T02:45:08.875+0000 7f381017b2c0 -1 osd.3 0 log_to_monitors true 2024-08-17T02:45:08.894 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-08-17T02:45:08.894+0000 7fc13b2752c0 -1 osd.1 0 log_to_monitors true 2024-08-17T02:45:08.901 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-08-17T02:45:08.901+0000 7fa0e64392c0 -1 osd.7 0 log_to_monitors true 2024-08-17T02:45:08.952 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-08-17T02:45:08.951+0000 7f397451f2c0 -1 osd.4 0 log_to_monitors true 2024-08-17T02:45:08.965 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-08-17T02:45:08.965+0000 7fbbe0e102c0 -1 osd.0 0 log_to_monitors true 2024-08-17T02:45:08.979 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-08-17T02:45:08.979+0000 7f325b2122c0 -1 osd.2 0 log_to_monitors true 2024-08-17T02:45:11.978 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-08-17T02:45:11.978+0000 7fbbda200640 -1 osd.0 0 waiting for initial osdmap 2024-08-17T02:45:11.983 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-08-17T02:45:11.983+0000 7fc136e00640 -1 osd.1 0 waiting for initial osdmap 2024-08-17T02:45:11.985 INFO:tasks.ceph.osd.0.smithi016.stderr:2024-08-17T02:45:11.986+0000 7fbbd1600640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:45:11.992 INFO:tasks.ceph.osd.1.smithi016.stderr:2024-08-17T02:45:11.992+0000 7fc12c400640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:45:12.028 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-08-17T02:45:12.028+0000 7f3809800640 -1 osd.3 0 waiting for initial osdmap 2024-08-17T02:45:12.031 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-08-17T02:45:12.032+0000 7f3254600640 -1 osd.2 0 waiting for initial osdmap 2024-08-17T02:45:12.034 INFO:tasks.ceph.osd.3.smithi016.stderr:2024-08-17T02:45:12.035+0000 7f3800c00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:45:12.037 INFO:tasks.ceph.osd.2.smithi016.stderr:2024-08-17T02:45:12.037+0000 7f324ba00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:45:12.475 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-08-17T02:45:12.474+0000 7feba0000640 -1 osd.5 0 waiting for initial osdmap 2024-08-17T02:45:12.482 INFO:tasks.ceph.osd.5.smithi142.stderr:2024-08-17T02:45:12.481+0000 7feb97400640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:45:12.499 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-08-17T02:45:12.498+0000 7fa0e2000640 -1 osd.7 0 waiting for initial osdmap 2024-08-17T02:45:12.506 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-08-17T02:45:12.505+0000 7f396dc00640 -1 osd.4 0 waiting for initial osdmap 2024-08-17T02:45:12.507 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-08-17T02:45:12.506+0000 7fa0d7600640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:45:12.513 INFO:tasks.ceph.osd.4.smithi142.stderr:2024-08-17T02:45:12.512+0000 7f3965000640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:45:12.582 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-08-17T02:45:12.580+0000 7f95fa400640 -1 osd.6 0 waiting for initial osdmap 2024-08-17T02:45:12.588 INFO:tasks.ceph.osd.6.smithi142.stderr:2024-08-17T02:45:12.587+0000 7f95efa00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-17T02:45:12.812 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:12.948 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.948+0000 7f646be00640 1 Processor -- start 2024-08-17T02:45:12.949 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.948+0000 7f646be00640 1 -- start start 2024-08-17T02:45:12.950 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.950+0000 7f646be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f646c154cc0 0x7f646c155090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:12.950 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.950+0000 7f646be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f646c0afab0 0x7f646c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:12.950 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.950+0000 7f646be00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646c0afef0 con 0x7f646c06e010 2024-08-17T02:45:12.950 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.950+0000 7f646be00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646c1555d0 con 0x7f646c154cc0 2024-08-17T02:45:12.950 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.950+0000 7f646be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646c06e9b0 con 0x7f646c0afab0 2024-08-17T02:45:12.950 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.950+0000 7f6463e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f646c06e010 0x7f646c0af3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43408/0 (socket says 172.21.15.16:43408) 2024-08-17T02:45:12.951 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.950+0000 7f6463e00640 1 -- 172.21.15.16:0/155300979 learned_addr learned my addr 172.21.15.16:0/155300979 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:12.951 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.950+0000 7f646aa00640 1 --2- 172.21.15.16:0/155300979 >> v2:172.21.15.142:3300/0 conn(0x7f646c154cc0 0x7f646c155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:12.951 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.950+0000 7f646b400640 1 --2- 172.21.15.16:0/155300979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f646c0afab0 0x7f646c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:12.952 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.950+0000 7f646a000640 1 -- 172.21.15.16:0/155300979 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435644528 0 0) 0x7f646c0afef0 con 0x7f646c06e010 2024-08-17T02:45:12.952 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.950+0000 7f646a000640 1 -- 172.21.15.16:0/155300979 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64500034e0 con 0x7f646c06e010 2024-08-17T02:45:12.952 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.951+0000 7f646aa00640 1 -- 172.21.15.16:0/155300979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f646c0afab0 msgr2=0x7f646c06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:12.952 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.951+0000 7f646aa00640 1 --2- 172.21.15.16:0/155300979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f646c0afab0 0x7f646c06e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:12.952 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.951+0000 7f646aa00640 1 -- 172.21.15.16:0/155300979 >> v1:172.21.15.16:6789/0 conn(0x7f646c06e010 legacy=0x7f646c0af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:12.952 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.951+0000 7f646aa00640 1 -- 172.21.15.16:0/155300979 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c0b0120 con 0x7f646c154cc0 2024-08-17T02:45:12.952 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.951+0000 7f646b400640 1 --2- 172.21.15.16:0/155300979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f646c0afab0 0x7f646c06e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:12.952 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.952+0000 7f646aa00640 1 --2- 172.21.15.16:0/155300979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646c154cc0 0x7f646c155090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f645c009850 tx=0x7f645c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfc29d00849bb72e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:12.952 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.952+0000 7f646a000640 1 -- 172.21.15.16:0/155300979 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c02ad00 con 0x7f646c154cc0 2024-08-17T02:45:12.953 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.952+0000 7f646a000640 1 -- 172.21.15.16:0/155300979 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c02ae60 con 0x7f646c154cc0 2024-08-17T02:45:12.953 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.952+0000 7f646a000640 1 -- 172.21.15.16:0/155300979 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c032650 con 0x7f646c154cc0 2024-08-17T02:45:12.953 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.952+0000 7f646be00640 1 -- 172.21.15.16:0/155300979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646c154cc0 msgr2=0x7f646c155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:12.954 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.952+0000 7f646be00640 1 --2- 172.21.15.16:0/155300979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646c154cc0 0x7f646c155090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f645c009850 tx=0x7f645c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:12.954 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.953+0000 7f646be00640 1 -- 172.21.15.16:0/155300979 shutdown_connections 2024-08-17T02:45:12.954 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.953+0000 7f646be00640 1 --2- 172.21.15.16:0/155300979 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f646c0afab0 0x7f646c06e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:12.954 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.953+0000 7f646be00640 1 --2- 172.21.15.16:0/155300979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646c154cc0 0x7f646c155090 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:12.954 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.953+0000 7f646be00640 1 -- 172.21.15.16:0/155300979 >> 172.21.15.16:0/155300979 conn(0x7f646c066d90 msgr2=0x7f646c067190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:12.954 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.953+0000 7f646be00640 1 -- 172.21.15.16:0/155300979 shutdown_connections 2024-08-17T02:45:12.954 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.953+0000 7f646be00640 1 -- 172.21.15.16:0/155300979 wait complete. 2024-08-17T02:45:12.954 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.954+0000 7f646be00640 1 Processor -- start 2024-08-17T02:45:12.955 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.954+0000 7f646be00640 1 -- start start 2024-08-17T02:45:12.957 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.955+0000 7f646be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f646c06e010 0x7f646c0c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:12.957 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.955+0000 7f646be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646c0afab0 0x7f646c0686f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:12.957 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.955+0000 7f646be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646c154cc0 0x7f646c068c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:12.957 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.955+0000 7f646be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0abe60 con 0x7f646c154cc0 2024-08-17T02:45:12.957 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.955+0000 7f646be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0ac140 con 0x7f646c0afab0 2024-08-17T02:45:12.957 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.955+0000 7f646be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646c14d670 con 0x7f646c06e010 2024-08-17T02:45:12.957 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.955+0000 7f646b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646c154cc0 0x7f646c068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:12.957 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f646b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646c154cc0 0x7f646c068c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53586/0 (socket says 172.21.15.16:53586) 2024-08-17T02:45:12.957 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f646b400640 1 -- 172.21.15.16:0/475802784 learned_addr learned my addr 172.21.15.16:0/475802784 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:12.957 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f646aa00640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f646c06e010 0x7f646c0c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:12.957 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f6463e00640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646c0afab0 0x7f646c0686f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:12.958 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f646b400640 1 -- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f646c06e010 msgr2=0x7f646c0c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:12.958 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f646b400640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f646c06e010 0x7f646c0c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:12.958 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f646b400640 1 -- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646c0afab0 msgr2=0x7f646c0686f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:12.958 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f646b400640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646c0afab0 0x7f646c0686f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:12.958 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f646b400640 1 -- 172.21.15.16:0/475802784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646400ab30 con 0x7f646c154cc0 2024-08-17T02:45:12.958 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f646b400640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646c154cc0 0x7f646c068c30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6464009f20 tx=0x7f646400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:12.958 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f6462000640 1 -- 172.21.15.16:0/475802784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464018070 con 0x7f646c154cc0 2024-08-17T02:45:12.958 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f6462000640 1 -- 172.21.15.16:0/475802784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6464010040 con 0x7f646c154cc0 2024-08-17T02:45:12.958 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f6462000640 1 -- 172.21.15.16:0/475802784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464014600 con 0x7f646c154cc0 2024-08-17T02:45:12.958 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.956+0000 7f646be00640 1 -- 172.21.15.16:0/475802784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c009390 con 0x7f646c154cc0 2024-08-17T02:45:12.958 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.957+0000 7f646be00640 1 -- 172.21.15.16:0/475802784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646c14dc30 con 0x7f646c154cc0 2024-08-17T02:45:12.959 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.958+0000 7f646be00640 1 -- 172.21.15.16:0/475802784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646c071100 con 0x7f646c154cc0 2024-08-17T02:45:12.959 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.958+0000 7f6462000640 1 -- 172.21.15.16:0/475802784 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 83364+0+0 (secure 0 0 0) 0x7f6464003020 con 0x7f646c154cc0 2024-08-17T02:45:12.959 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.959+0000 7f6462000640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f64480650c0 0x7f6448067570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:12.959 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.959+0000 7f6462000640 1 -- 172.21.15.16:0/475802784 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7f6464081c80 con 0x7f646c154cc0 2024-08-17T02:45:12.960 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.960+0000 7f646aa00640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f64480650c0 0x7f6448067570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:12.960 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.961+0000 7f646aa00640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f64480650c0 0x7f6448067570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f645c034000 tx=0x7f645c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:12.964 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:12.964+0000 7f6462000640 1 -- 172.21.15.16:0/475802784 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6464050c70 con 0x7f646c154cc0 2024-08-17T02:45:13.158 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.158+0000 7f646be00640 1 -- 172.21.15.16:0/475802784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f646c06e470 con 0x7f646c154cc0 2024-08-17T02:45:13.159 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.159+0000 7f6462000640 1 -- 172.21.15.16:0/475802784 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10396 (secure 0 0 0) 0x7f6464054290 con 0x7f646c154cc0 2024-08-17T02:45:13.160 INFO:teuthology.misc.health.smithi016.stdout: 2024-08-17T02:45:13.160 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":13,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:12.982747+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:13.161 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.161+0000 7f646be00640 1 -- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f64480650c0 msgr2=0x7f6448067570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.161 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.161+0000 7f646be00640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f64480650c0 0x7f6448067570 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f645c034000 tx=0x7f645c032000 comp rx=0 tx=0).stop 2024-08-17T02:45:13.162 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.161+0000 7f646be00640 1 -- 172.21.15.16:0/475802784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646c154cc0 msgr2=0x7f646c068c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.162 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.161+0000 7f646be00640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646c154cc0 0x7f646c068c30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6464009f20 tx=0x7f646400a890 comp rx=0 tx=0).stop 2024-08-17T02:45:13.162 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.161+0000 7f646be00640 1 -- 172.21.15.16:0/475802784 shutdown_connections 2024-08-17T02:45:13.162 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.161+0000 7f646be00640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f64480650c0 0x7f6448067570 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.162 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.161+0000 7f646be00640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f646c154cc0 0x7f646c068c30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.162 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.161+0000 7f646be00640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f646c0afab0 0x7f646c0686f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.162 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.161+0000 7f646be00640 1 --2- 172.21.15.16:0/475802784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f646c06e010 0x7f646c0c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.162 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.161+0000 7f646be00640 1 -- 172.21.15.16:0/475802784 >> 172.21.15.16:0/475802784 conn(0x7f646c066d90 msgr2=0x7f646c153b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:13.162 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.161+0000 7f646be00640 1 -- 172.21.15.16:0/475802784 shutdown_connections 2024-08-17T02:45:13.162 INFO:teuthology.misc.health.smithi016.stderr:2024-08-17T02:45:13.161+0000 7f646be00640 1 -- 172.21.15.16:0/475802784 wait complete. 2024-08-17T02:45:13.175 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-08-17T02:45:13.175 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-08-17T02:45:13.175 INFO:tasks.ceph.mds.a:Restarting daemon 2024-08-17T02:45:13.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-08-17T02:45:13.180 INFO:tasks.ceph.mds.a:Started 2024-08-17T02:45:13.181 INFO:tasks.ceph.mds.c:Restarting daemon 2024-08-17T02:45:13.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-08-17T02:45:13.185 INFO:tasks.ceph.mds.c:Started 2024-08-17T02:45:13.185 INFO:tasks.ceph.mds.b:Restarting daemon 2024-08-17T02:45:13.185 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-08-17T02:45:13.192 INFO:tasks.ceph.mds.b:Started 2024-08-17T02:45:13.193 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-17T02:45:13.193 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-17T02:45:13.197 INFO:tasks.ceph.mds.d:Started 2024-08-17T02:45:13.197 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-08-17T02:45:13.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:13.306 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-08-17T02:45:13.308 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-08-17T02:45:13.320 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-08-17T02:45:13.363 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-08-17T02:45:13.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.358+0000 7f78eb000640 1 Processor -- start 2024-08-17T02:45:13.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.358+0000 7f78eb000640 1 -- start start 2024-08-17T02:45:13.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.359+0000 7f78eb000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f78e4159ca0 0x7f78e40ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.359+0000 7f78eb000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78e40ca8a0 0x7f78e40cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.359+0000 7f78eb000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e40cadc0 con 0x7f78e40ccad0 2024-08-17T02:45:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.359+0000 7f78eb000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e40cd6e0 con 0x7f78e4159ca0 2024-08-17T02:45:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.359+0000 7f78eb000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e40ccf70 con 0x7f78e40ca8a0 2024-08-17T02:45:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.359+0000 7f78e9c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f78e4159ca0 0x7f78e40ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.359+0000 7f78e9200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78e40ca8a0 0x7f78e40cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.359+0000 7f78e9c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f78e4159ca0 0x7f78e40ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:41966/0 (socket says 172.21.15.16:41966) 2024-08-17T02:45:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.359+0000 7f78e9c00640 1 -- 172.21.15.16:0/3966541711 learned_addr learned my addr 172.21.15.16:0/3966541711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.359+0000 7f78e9200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78e40ca8a0 0x7f78e40cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:41592/0 (socket says 172.21.15.16:41592) 2024-08-17T02:45:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.359+0000 7f78ea600640 1 --1- 172.21.15.16:0/3966541711 >> v1:172.21.15.142:6789/0 conn(0x7f78e40ccad0 0x7f78e40cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:36856/0 (socket says 172.21.15.16:36856) 2024-08-17T02:45:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.360+0000 7f78e9c00640 1 -- 172.21.15.16:0/3966541711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78e40ca8a0 msgr2=0x7f78e40cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.360+0000 7f78e9c00640 1 --2- 172.21.15.16:0/3966541711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78e40ca8a0 0x7f78e40cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.360+0000 7f78e9c00640 1 -- 172.21.15.16:0/3966541711 >> v1:172.21.15.142:6789/0 conn(0x7f78e40ccad0 legacy=0x7f78e40cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.360+0000 7f78e9c00640 1 -- 172.21.15.16:0/3966541711 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e40cb060 con 0x7f78e4159ca0 2024-08-17T02:45:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.360+0000 7f78e3e00640 1 -- 172.21.15.16:0/3966541711 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144487561 0 0) 0x7f78e40cadc0 con 0x7f78e40ccad0 2024-08-17T02:45:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.361+0000 7f78e9c00640 1 --2- 172.21.15.16:0/3966541711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e4159ca0 0x7f78e40ca360 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009910 tx=0x7f78d80091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf6a3770af15ab33 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.361+0000 7f78e3e00640 1 -- 172.21.15.16:0/3966541711 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d802ae40 con 0x7f78e4159ca0 2024-08-17T02:45:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.361+0000 7f78e3e00640 1 -- 172.21.15.16:0/3966541711 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d8032440 con 0x7f78e4159ca0 2024-08-17T02:45:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.361+0000 7f78e3e00640 1 -- 172.21.15.16:0/3966541711 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d80327e0 con 0x7f78e4159ca0 2024-08-17T02:45:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.362+0000 7f78eb000640 1 -- 172.21.15.16:0/3966541711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e4159ca0 msgr2=0x7f78e40ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.362+0000 7f78eb000640 1 --2- 172.21.15.16:0/3966541711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e4159ca0 0x7f78e40ca360 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009910 tx=0x7f78d80091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.362+0000 7f78eb000640 1 -- 172.21.15.16:0/3966541711 shutdown_connections 2024-08-17T02:45:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.362+0000 7f78eb000640 1 --2- 172.21.15.16:0/3966541711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78e40ca8a0 0x7f78e40cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.362+0000 7f78eb000640 1 --2- 172.21.15.16:0/3966541711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e4159ca0 0x7f78e40ca360 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.362+0000 7f78eb000640 1 -- 172.21.15.16:0/3966541711 >> 172.21.15.16:0/3966541711 conn(0x7f78e4098e00 msgr2=0x7f78e406eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.362+0000 7f78eb000640 1 -- 172.21.15.16:0/3966541711 shutdown_connections 2024-08-17T02:45:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.362+0000 7f78eb000640 1 -- 172.21.15.16:0/3966541711 wait complete. 2024-08-17T02:45:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.363+0000 7f78eb000640 1 Processor -- start 2024-08-17T02:45:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.363+0000 7f78eb000640 1 -- start start 2024-08-17T02:45:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.364+0000 7f78eb000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e40ca8a0 0x7f78e4099770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.364+0000 7f78eb000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e40ccad0 0x7f78e409b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.364+0000 7f78eb000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78e4159ca0 0x7f78e409b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.364+0000 7f78eb000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e40c60c0 con 0x7f78e40ca8a0 2024-08-17T02:45:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.364+0000 7f78eb000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e40c54b0 con 0x7f78e40ccad0 2024-08-17T02:45:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.364+0000 7f78eb000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e40c5790 con 0x7f78e4159ca0 2024-08-17T02:45:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78e9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e40ca8a0 0x7f78e4099770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78e9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e40ca8a0 0x7f78e4099770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53652/0 (socket says 172.21.15.16:53652) 2024-08-17T02:45:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78e9c00640 1 -- 172.21.15.16:0/2268606172 learned_addr learned my addr 172.21.15.16:0/2268606172 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78ea600640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78e4159ca0 0x7f78e409b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78e9200640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e40ccad0 0x7f78e409b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78e9c00640 1 -- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78e4159ca0 msgr2=0x7f78e409b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78e9c00640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78e4159ca0 0x7f78e409b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78e9c00640 1 -- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e40ccad0 msgr2=0x7f78e409b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78e9c00640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e40ccad0 0x7f78e409b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78e9c00640 1 -- 172.21.15.16:0/2268606172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d8009450 con 0x7f78e40ca8a0 2024-08-17T02:45:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78e9200640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e40ccad0 0x7f78e409b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78ea600640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78e4159ca0 0x7f78e409b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.365+0000 7f78e9c00640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e40ca8a0 0x7f78e4099770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f78d802a840 tx=0x7f78d8032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.366+0000 7f78e0c00640 1 -- 172.21.15.16:0/2268606172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8000c40 con 0x7f78e40ca8a0 2024-08-17T02:45:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.366+0000 7f78e0c00640 1 -- 172.21.15.16:0/2268606172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d8031a60 con 0x7f78e40ca8a0 2024-08-17T02:45:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.366+0000 7f78eb000640 1 -- 172.21.15.16:0/2268606172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e40c5a70 con 0x7f78e40ca8a0 2024-08-17T02:45:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.366+0000 7f78e0c00640 1 -- 172.21.15.16:0/2268606172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8031e60 con 0x7f78e40ca8a0 2024-08-17T02:45:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.366+0000 7f78eb000640 1 -- 172.21.15.16:0/2268606172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e40c51c0 con 0x7f78e40ca8a0 2024-08-17T02:45:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.367+0000 7f78eb000640 1 -- 172.21.15.16:0/2268606172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e4071050 con 0x7f78e40ca8a0 2024-08-17T02:45:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.368+0000 7f78e0c00640 1 -- 172.21.15.16:0/2268606172 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 83364+0+0 (secure 0 0 0) 0x7f78d8047450 con 0x7f78e40ca8a0 2024-08-17T02:45:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.368+0000 7f78e0c00640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f78c4065190 0x7f78c4067640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.369+0000 7f78e0c00640 1 -- 172.21.15.16:0/2268606172 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f78d802ae40 con 0x7f78e40ca8a0 2024-08-17T02:45:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.369+0000 7f78e9200640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f78c4065190 0x7f78c4067640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.370+0000 7f78e9200640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f78c4065190 0x7f78c4067640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78e40ca820 tx=0x7f78d00090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.373+0000 7f78e0c00640 1 -- 172.21.15.16:0/2268606172 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f78d8070660 con 0x7f78e40ca8a0 2024-08-17T02:45:13.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.560+0000 7f78eb000640 1 -- 172.21.15.16:0/2268606172 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78e4089bd0 con 0x7f78e40ca8a0 2024-08-17T02:45:13.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.560+0000 7f78e0c00640 1 -- 172.21.15.16:0/2268606172 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10396 (secure 0 0 0) 0x7f78d8037090 con 0x7f78e40ca8a0 2024-08-17T02:45:13.561 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:13.561 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:12.982747+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:13.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.563+0000 7f78eb000640 1 -- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f78c4065190 msgr2=0x7f78c4067640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.563+0000 7f78eb000640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f78c4065190 0x7f78c4067640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78e40ca820 tx=0x7f78d00090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.563+0000 7f78eb000640 1 -- 172.21.15.16:0/2268606172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e40ca8a0 msgr2=0x7f78e4099770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.563+0000 7f78eb000640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e40ca8a0 0x7f78e4099770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f78d802a840 tx=0x7f78d8032000 comp rx=0 tx=0).stop 2024-08-17T02:45:13.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.563+0000 7f78eb000640 1 -- 172.21.15.16:0/2268606172 shutdown_connections 2024-08-17T02:45:13.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.563+0000 7f78eb000640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f78c4065190 0x7f78c4067640 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.563+0000 7f78eb000640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78e4159ca0 0x7f78e409b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.563+0000 7f78eb000640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78e40ccad0 0x7f78e409b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.563+0000 7f78eb000640 1 --2- 172.21.15.16:0/2268606172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78e40ca8a0 0x7f78e4099770 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.563+0000 7f78eb000640 1 -- 172.21.15.16:0/2268606172 >> 172.21.15.16:0/2268606172 conn(0x7f78e4098e00 msgr2=0x7f78e41582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:13.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.563+0000 7f78eb000640 1 -- 172.21.15.16:0/2268606172 shutdown_connections 2024-08-17T02:45:13.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.563+0000 7f78eb000640 1 -- 172.21.15.16:0/2268606172 wait complete. 2024-08-17T02:45:13.578 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-08-17T02:45:13.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:45:13.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.730+0000 7febde600640 1 Processor -- start 2024-08-17T02:45:13.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.730+0000 7febde600640 1 -- start start 2024-08-17T02:45:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.731+0000 7febde600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 0x7febd80c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.731+0000 7febde600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd80cb7c0 con 0x7febd80cb340 2024-08-17T02:45:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.731+0000 7febde600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd80ceb20 con 0x7febd8158ce0 2024-08-17T02:45:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.731+0000 7febde600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd80cf0c0 con 0x7febd80d08d0 2024-08-17T02:45:13.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.731+0000 7febd7e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7febd80cb340 0x7febd80c5520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43420/0 (socket says 172.21.15.16:43420) 2024-08-17T02:45:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.731+0000 7febd7e00640 1 -- 172.21.15.16:0/1742089614 learned_addr learned my addr 172.21.15.16:0/1742089614 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febcfe00640 1 --2- 172.21.15.16:0/1742089614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 0x7febd80c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285807484 0 0) 0x7febd80cb7c0 con 0x7febd80cb340 2024-08-17T02:45:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febbc0034e0 con 0x7febd80cb340 2024-08-17T02:45:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2899048826 0 0) 0x7febbc0034e0 con 0x7febd80cb340 2024-08-17T02:45:13.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 msgr2=0x7febd80c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 --2- 172.21.15.16:0/1742089614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 0x7febd80c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 >> v1:172.21.15.142:6789/0 conn(0x7febd8158ce0 legacy=0x7febd80cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd80cede0 con 0x7febd80cb340 2024-08-17T02:45:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7febd00017f0 con 0x7febd80cb340 2024-08-17T02:45:13.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 >> v1:172.21.15.16:6789/0 conn(0x7febd80cb340 legacy=0x7febd80c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 --2- 172.21.15.16:0/1742089614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febbc003a00 0x7febbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 --2- 172.21.15.16:0/1742089614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 0x7febbc004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 --2- 172.21.15.16:0/1742089614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febbc004cc0 0x7febbc005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd80cf0c0 con 0x7febbc003a00 2024-08-17T02:45:13.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc00ba80 con 0x7febbc004cc0 2024-08-17T02:45:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc00bc40 con 0x7febd80d08d0 2024-08-17T02:45:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd7400640 1 --2- 172.21.15.16:0/1742089614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febbc003a00 0x7febbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febcfe00640 1 --2- 172.21.15.16:0/1742089614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 0x7febbc004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.732+0000 7febd7e00640 1 --2- 172.21.15.16:0/1742089614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febbc004cc0 0x7febbc005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.733+0000 7febcfe00640 1 -- 172.21.15.16:0/1742089614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febbc004cc0 msgr2=0x7febbc005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.733+0000 7febcfe00640 1 --2- 172.21.15.16:0/1742089614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febbc004cc0 0x7febbc005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.733+0000 7febcfe00640 1 -- 172.21.15.16:0/1742089614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febbc003a00 msgr2=0x7febbc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.733+0000 7febcfe00640 1 --2- 172.21.15.16:0/1742089614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febbc003a00 0x7febbc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.733+0000 7febcfe00640 1 -- 172.21.15.16:0/1742089614 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc0034e0 con 0x7febd80d08d0 2024-08-17T02:45:13.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.734+0000 7febcfe00640 1 --2- 172.21.15.16:0/1742089614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 0x7febbc004780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7febc0009b70 tx=0x7febc00090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6b19f9c340907ec server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.734+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0031400 con 0x7febd80d08d0 2024-08-17T02:45:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.734+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc002d050 con 0x7febd80d08d0 2024-08-17T02:45:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.734+0000 7febd6a00640 1 -- 172.21.15.16:0/1742089614 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0031790 con 0x7febd80d08d0 2024-08-17T02:45:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.735+0000 7febde600640 1 -- 172.21.15.16:0/1742089614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 msgr2=0x7febbc004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.735+0000 7febde600640 1 --2- 172.21.15.16:0/1742089614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 0x7febbc004780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7febc0009b70 tx=0x7febc00090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.735+0000 7febd7e00640 1 -- 172.21.15.16:0/1742089614 reap_dead start 2024-08-17T02:45:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.735+0000 7febde600640 1 -- 172.21.15.16:0/1742089614 shutdown_connections 2024-08-17T02:45:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.735+0000 7febde600640 1 -- 172.21.15.16:0/1742089614 >> 172.21.15.16:0/1742089614 conn(0x7febd80cfdc0 msgr2=0x7febd80d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.735+0000 7febde600640 1 -- 172.21.15.16:0/1742089614 shutdown_connections 2024-08-17T02:45:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.735+0000 7febde600640 1 -- 172.21.15.16:0/1742089614 wait complete. 2024-08-17T02:45:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.736+0000 7febde600640 1 Processor -- start 2024-08-17T02:45:13.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.736+0000 7febde600640 1 -- start start 2024-08-17T02:45:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.737+0000 7febde600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 0x7febd80c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.737+0000 7febde600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd80c8170 0x7febd8099970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.737+0000 7febde600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febd816c250 0x7febd816c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.737+0000 7febde600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd816cc00 con 0x7febd80c8170 2024-08-17T02:45:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.737+0000 7febde600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd80c8f40 con 0x7febd816c250 2024-08-17T02:45:13.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.737+0000 7febde600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd80b2cf0 con 0x7febd80d08d0 2024-08-17T02:45:13.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.737+0000 7febcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd80c8170 0x7febd8099970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd80c8170 0x7febd8099970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53668/0 (socket says 172.21.15.16:53668) 2024-08-17T02:45:13.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febcfe00640 1 -- 172.21.15.16:0/4259971211 learned_addr learned my addr 172.21.15.16:0/4259971211 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:13.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febd7400640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 0x7febd80c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febd7e00640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febd816c250 0x7febd816c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febcfe00640 1 -- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 msgr2=0x7febd80c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febcfe00640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 0x7febd80c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febcfe00640 1 -- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febd816c250 msgr2=0x7febd816c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febcfe00640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febd816c250 0x7febd816c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febcfe00640 1 -- 172.21.15.16:0/4259971211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd80b2fd0 con 0x7febd80c8170 2024-08-17T02:45:13.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febcfe00640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd80c8170 0x7febd8099970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7febc002ab10 tx=0x7febc0035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febcea00640 1 -- 172.21.15.16:0/4259971211 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc003c7d0 con 0x7febd80c8170 2024-08-17T02:45:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febcea00640 1 -- 172.21.15.16:0/4259971211 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc003c930 con 0x7febd80c8170 2024-08-17T02:45:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febcea00640 1 -- 172.21.15.16:0/4259971211 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc003ccb0 con 0x7febd80c8170 2024-08-17T02:45:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.738+0000 7febde600640 1 -- 172.21.15.16:0/4259971211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd80b32b0 con 0x7febd80c8170 2024-08-17T02:45:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.739+0000 7febde600640 1 -- 172.21.15.16:0/4259971211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd80c8bd0 con 0x7febd80c8170 2024-08-17T02:45:13.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.740+0000 7febde600640 1 -- 172.21.15.16:0/4259971211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd8064db0 con 0x7febd80c8170 2024-08-17T02:45:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.740+0000 7febcea00640 1 -- 172.21.15.16:0/4259971211 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 83364+0+0 (secure 0 0 0) 0x7febc0030420 con 0x7febd80c8170 2024-08-17T02:45:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.741+0000 7febcea00640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7febb40650c0 0x7febb4067570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.741+0000 7febcea00640 1 -- 172.21.15.16:0/4259971211 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7febc00a3230 con 0x7febd80c8170 2024-08-17T02:45:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.741+0000 7febd7400640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7febb40650c0 0x7febb4067570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:13.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.742+0000 7febd7400640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7febb40650c0 0x7febb4067570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7febc8007fd0 tx=0x7febc8007ca0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:13.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.745+0000 7febcea00640 1 -- 172.21.15.16:0/4259971211 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7febc0071800 con 0x7febd80c8170 2024-08-17T02:45:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.965+0000 7febde600640 1 -- 172.21.15.16:0/4259971211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febd8069260 con 0x7febd80c8170 2024-08-17T02:45:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.965+0000 7febcea00640 1 -- 172.21.15.16:0/4259971211 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1256 (secure 0 0 0) 0x7febc0074e20 con 0x7febd80c8170 2024-08-17T02:45:13.965 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:13.966 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3179355872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3179355872},{"type":"v1","addr":"172.21.15.16:6833","nonce":3179355872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-08-17T02:45:13.966 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-08-17T02:45:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.967+0000 7febde600640 1 -- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7febb40650c0 msgr2=0x7febb4067570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.967+0000 7febde600640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7febb40650c0 0x7febb4067570 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7febc8007fd0 tx=0x7febc8007ca0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.967+0000 7febde600640 1 -- 172.21.15.16:0/4259971211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd80c8170 msgr2=0x7febd8099970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.967+0000 7febde600640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd80c8170 0x7febd8099970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7febc002ab10 tx=0x7febc0035000 comp rx=0 tx=0).stop 2024-08-17T02:45:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.967+0000 7febde600640 1 -- 172.21.15.16:0/4259971211 shutdown_connections 2024-08-17T02:45:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.967+0000 7febde600640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7febb40650c0 0x7febb4067570 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.967+0000 7febde600640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febd816c250 0x7febd816c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.967+0000 7febde600640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd80c8170 0x7febd8099970 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.967+0000 7febde600640 1 --2- 172.21.15.16:0/4259971211 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7febd80d08d0 0x7febd80c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.967+0000 7febde600640 1 -- 172.21.15.16:0/4259971211 >> 172.21.15.16:0/4259971211 conn(0x7febd80cfdc0 msgr2=0x7febd815a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.967+0000 7febde600640 1 -- 172.21.15.16:0/4259971211 shutdown_connections 2024-08-17T02:45:13.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:13.968+0000 7febde600640 1 -- 172.21.15.16:0/4259971211 wait complete. 2024-08-17T02:45:15.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:45:16.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.166+0000 7f6e01a00640 1 Processor -- start 2024-08-17T02:45:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.166+0000 7f6e01a00640 1 -- start start 2024-08-17T02:45:16.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.167+0000 7f6e01a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0ada10 0x7f6dfc0b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.167+0000 7f6e01a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6dfc0b1120 0x7f6dfc0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.167+0000 7f6e01a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc0d0ab0 con 0x7f6dfc0b36e0 2024-08-17T02:45:16.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.167+0000 7f6e01a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dfc0b00c0 con 0x7f6dfc0b1120 2024-08-17T02:45:16.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.167+0000 7f6e01a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0b0360 con 0x7f6dfc0ada10 2024-08-17T02:45:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.168+0000 7f6dfaa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0ada10 0x7f6dfc0b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.168+0000 7f6df3e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6dfc0b1120 0x7f6dfc0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.168+0000 7f6dfaa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0ada10 0x7f6dfc0b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:59140/0 (socket says 172.21.15.16:59140) 2024-08-17T02:45:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.168+0000 7f6dfaa00640 1 -- 172.21.15.16:0/1547926134 learned_addr learned my addr 172.21.15.16:0/1547926134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.168+0000 7f6dfb400640 1 --1- 172.21.15.16:0/1547926134 >> v1:172.21.15.142:6789/0 conn(0x7f6dfc0b36e0 0x7f6dfc0adde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:39150/0 (socket says 172.21.15.16:39150) 2024-08-17T02:45:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.168+0000 7f6df3e00640 1 -- 172.21.15.16:0/1547926134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0ada10 msgr2=0x7f6dfc0b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.168+0000 7f6df3e00640 1 --2- 172.21.15.16:0/1547926134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0ada10 0x7f6dfc0b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.168+0000 7f6df3e00640 1 -- 172.21.15.16:0/1547926134 >> v1:172.21.15.142:6789/0 conn(0x7f6dfc0b36e0 legacy=0x7f6dfc0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.168+0000 7f6df3e00640 1 -- 172.21.15.16:0/1547926134 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc0b0640 con 0x7f6dfc0b1120 2024-08-17T02:45:16.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.168+0000 7f6dfaa00640 1 --2- 172.21.15.16:0/1547926134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0ada10 0x7f6dfc0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:16.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.168+0000 7f6dfa000640 1 -- 172.21.15.16:0/1547926134 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203602141 0 0) 0x7f6dfc0d0ab0 con 0x7f6dfc0b36e0 2024-08-17T02:45:16.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.169+0000 7f6df3e00640 1 --2- 172.21.15.16:0/1547926134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc0b1120 0x7f6dfc0b31a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009850 tx=0x7f6de40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c1761fcb5e73903 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:16.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.170+0000 7f6dfa000640 1 -- 172.21.15.16:0/1547926134 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de402ad90 con 0x7f6dfc0b1120 2024-08-17T02:45:16.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.170+0000 7f6dfa000640 1 -- 172.21.15.16:0/1547926134 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de4032440 con 0x7f6dfc0b1120 2024-08-17T02:45:16.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.170+0000 7f6dfa000640 1 -- 172.21.15.16:0/1547926134 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de40327e0 con 0x7f6dfc0b1120 2024-08-17T02:45:16.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.170+0000 7f6e01a00640 1 -- 172.21.15.16:0/1547926134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc0b1120 msgr2=0x7f6dfc0b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.170+0000 7f6e01a00640 1 --2- 172.21.15.16:0/1547926134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc0b1120 0x7f6dfc0b31a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009850 tx=0x7f6de40090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.170+0000 7f6e01a00640 1 -- 172.21.15.16:0/1547926134 shutdown_connections 2024-08-17T02:45:16.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.170+0000 7f6e01a00640 1 --2- 172.21.15.16:0/1547926134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc0b1120 0x7f6dfc0b31a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.170+0000 7f6e01a00640 1 --2- 172.21.15.16:0/1547926134 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0ada10 0x7f6dfc0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.170+0000 7f6e01a00640 1 -- 172.21.15.16:0/1547926134 >> 172.21.15.16:0/1547926134 conn(0x7f6dfc0cfdf0 msgr2=0x7f6dfc0d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:16.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.170+0000 7f6e01a00640 1 -- 172.21.15.16:0/1547926134 shutdown_connections 2024-08-17T02:45:16.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.170+0000 7f6e01a00640 1 -- 172.21.15.16:0/1547926134 wait complete. 2024-08-17T02:45:16.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.172+0000 7f6e01a00640 1 Processor -- start 2024-08-17T02:45:16.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.172+0000 7f6e01a00640 1 -- start start 2024-08-17T02:45:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.173+0000 7f6e01a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dfc0ada10 0x7f6dfc14bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.173+0000 7f6e01a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0b1120 0x7f6dfc159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.173+0000 7f6e01a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc0b36e0 0x7f6dfc157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.173+0000 7f6e01a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc15a4b0 con 0x7f6dfc0ada10 2024-08-17T02:45:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.173+0000 7f6e01a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc159ef0 con 0x7f6dfc0b36e0 2024-08-17T02:45:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.173+0000 7f6e01a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc15a1d0 con 0x7f6dfc0b1120 2024-08-17T02:45:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.173+0000 7f6dfaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dfc0ada10 0x7f6dfc14bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.173+0000 7f6dfaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dfc0ada10 0x7f6dfc14bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32906/0 (socket says 172.21.15.16:32906) 2024-08-17T02:45:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.173+0000 7f6dfaa00640 1 -- 172.21.15.16:0/1940302233 learned_addr learned my addr 172.21.15.16:0/1940302233 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:16.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.173+0000 7f6dfb400640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc0b36e0 0x7f6dfc157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.173+0000 7f6df3e00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0b1120 0x7f6dfc159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.174+0000 7f6dfaa00640 1 -- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0b1120 msgr2=0x7f6dfc159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.174+0000 7f6dfaa00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0b1120 0x7f6dfc159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.174+0000 7f6dfaa00640 1 -- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc0b36e0 msgr2=0x7f6dfc157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.174+0000 7f6dfaa00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc0b36e0 0x7f6dfc157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.174+0000 7f6dfaa00640 1 -- 172.21.15.16:0/1940302233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec009450 con 0x7f6dfc0ada10 2024-08-17T02:45:16.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.174+0000 7f6df3e00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0b1120 0x7f6dfc159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:16.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.174+0000 7f6dfaa00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dfc0ada10 0x7f6dfc14bb40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00ec80 tx=0x7f6dec0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:16.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.174+0000 7f6df2000640 1 -- 172.21.15.16:0/1940302233 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec009730 con 0x7f6dfc0ada10 2024-08-17T02:45:16.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.174+0000 7f6df2000640 1 -- 172.21.15.16:0/1940302233 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec00eec0 con 0x7f6dfc0ada10 2024-08-17T02:45:16.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.174+0000 7f6e01a00640 1 -- 172.21.15.16:0/1940302233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de4009390 con 0x7f6dfc0ada10 2024-08-17T02:45:16.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.174+0000 7f6df2000640 1 -- 172.21.15.16:0/1940302233 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec00cdf0 con 0x7f6dfc0ada10 2024-08-17T02:45:16.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.174+0000 7f6e01a00640 1 -- 172.21.15.16:0/1940302233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc15cd50 con 0x7f6dfc0ada10 2024-08-17T02:45:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.175+0000 7f6e01a00640 1 -- 172.21.15.16:0/1940302233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc064db0 con 0x7f6dfc0ada10 2024-08-17T02:45:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.176+0000 7f6df2000640 1 -- 172.21.15.16:0/1940302233 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 83364+0+0 (secure 0 0 0) 0x7f6dec013680 con 0x7f6dfc0ada10 2024-08-17T02:45:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.177+0000 7f6df2000640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6dd80650c0 0x7f6dd8067570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.177+0000 7f6df2000640 1 -- 172.21.15.16:0/1940302233 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6dec082410 con 0x7f6dfc0ada10 2024-08-17T02:45:16.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.178+0000 7f6df3e00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6dd80650c0 0x7f6dd8067570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.178+0000 7f6df3e00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6dd80650c0 0x7f6dd8067570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6de4034000 tx=0x7f6de4032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.182+0000 7f6df2000640 1 -- 172.21.15.16:0/1940302233 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6dec050910 con 0x7f6dfc0ada10 2024-08-17T02:45:16.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.416+0000 7f6e01a00640 1 -- 172.21.15.16:0/1940302233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dfc0d0bb0 con 0x7f6dfc0ada10 2024-08-17T02:45:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.417+0000 7f6df2000640 1 -- 172.21.15.16:0/1940302233 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7f6dec017de0 con 0x7f6dfc0ada10 2024-08-17T02:45:16.417 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:16.417 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/111049205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":111049205},{"type":"v1","addr":"172.21.15.142:6836","nonce":111049205}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2692359383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2692359383},{"type":"v1","addr":"172.21.15.142:6837","nonce":2692359383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3179355872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3179355872},{"type":"v1","addr":"172.21.15.16:6833","nonce":3179355872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4266,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/2140506421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2140506421},{"type":"v1","addr":"172.21.15.16:6835","nonce":2140506421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-08-17T02:45:16.417 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2024-08-17T02:45:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.420+0000 7f6e01a00640 1 -- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6dd80650c0 msgr2=0x7f6dd8067570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.420+0000 7f6e01a00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6dd80650c0 0x7f6dd8067570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6de4034000 tx=0x7f6de4032000 comp rx=0 tx=0).stop 2024-08-17T02:45:16.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.420+0000 7f6e01a00640 1 -- 172.21.15.16:0/1940302233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dfc0ada10 msgr2=0x7f6dfc14bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.420+0000 7f6e01a00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dfc0ada10 0x7f6dfc14bb40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00ec80 tx=0x7f6dec0091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.420+0000 7f6e01a00640 1 -- 172.21.15.16:0/1940302233 shutdown_connections 2024-08-17T02:45:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.420+0000 7f6e01a00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6dd80650c0 0x7f6dd8067570 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.420+0000 7f6e01a00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6dfc0b36e0 0x7f6dfc157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.420+0000 7f6e01a00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6dfc0b1120 0x7f6dfc159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.420+0000 7f6e01a00640 1 --2- 172.21.15.16:0/1940302233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6dfc0ada10 0x7f6dfc14bb40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.420+0000 7f6e01a00640 1 -- 172.21.15.16:0/1940302233 >> 172.21.15.16:0/1940302233 conn(0x7f6dfc0cfdf0 msgr2=0x7f6dfc068190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.420+0000 7f6e01a00640 1 -- 172.21.15.16:0/1940302233 shutdown_connections 2024-08-17T02:45:16.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.420+0000 7f6e01a00640 1 -- 172.21.15.16:0/1940302233 wait complete. 2024-08-17T02:45:16.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.588+0000 7f8549600640 1 Processor -- start 2024-08-17T02:45:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.588+0000 7f8549600640 1 -- start start 2024-08-17T02:45:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.589+0000 7f8549600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8544158ce0 0x7f85440c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.589+0000 7f8549600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f85440d08d0 0x7f85441590b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.589+0000 7f8549600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85440cac30 0x7f85440cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.589+0000 7f8549600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85440cf0c0 con 0x7f85440cac30 2024-08-17T02:45:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.589+0000 7f8549600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8544089000 con 0x7f85440d08d0 2024-08-17T02:45:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.589+0000 7f8549600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85440ceb20 con 0x7f8544158ce0 2024-08-17T02:45:16.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.590+0000 7f8543400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85440cac30 0x7f85440cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.590+0000 7f8543400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85440cac30 0x7f85440cb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32922/0 (socket says 172.21.15.16:32922) 2024-08-17T02:45:16.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.590+0000 7f8543400640 1 -- 172.21.15.16:0/4141058895 learned_addr learned my addr 172.21.15.16:0/4141058895 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:16.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.590+0000 7f8542000640 1 --2- 172.21.15.16:0/4141058895 >> v2:172.21.15.142:3300/0 conn(0x7f85440d08d0 0x7f85441590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.590+0000 7f8542a00640 1 --2- 172.21.15.16:0/4141058895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8544158ce0 0x7f85440c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.590+0000 7f8543400640 1 -- 172.21.15.16:0/4141058895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8544158ce0 msgr2=0x7f85440c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:16.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.590+0000 7f8543400640 1 --2- 172.21.15.16:0/4141058895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8544158ce0 0x7f85440c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.590+0000 7f8543400640 1 -- 172.21.15.16:0/4141058895 >> v2:172.21.15.142:3300/0 conn(0x7f85440d08d0 msgr2=0x7f85441590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:16.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.590+0000 7f8543400640 1 --2- 172.21.15.16:0/4141058895 >> v2:172.21.15.142:3300/0 conn(0x7f85440d08d0 0x7f85441590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.590+0000 7f8543400640 1 -- 172.21.15.16:0/4141058895 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85440cede0 con 0x7f85440cac30 2024-08-17T02:45:16.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.591+0000 7f8543400640 1 --2- 172.21.15.16:0/4141058895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440cac30 0x7f85440cb0b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8544064640 tx=0x7f853c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09d3ade629656b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:16.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.592+0000 7f8541600640 1 -- 172.21.15.16:0/4141058895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c031400 con 0x7f85440cac30 2024-08-17T02:45:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.592+0000 7f8541600640 1 -- 172.21.15.16:0/4141058895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853c02d050 con 0x7f85440cac30 2024-08-17T02:45:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.592+0000 7f8541600640 1 -- 172.21.15.16:0/4141058895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c0317d0 con 0x7f85440cac30 2024-08-17T02:45:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.592+0000 7f8549600640 1 -- 172.21.15.16:0/4141058895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440cac30 msgr2=0x7f85440cb0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.592+0000 7f8549600640 1 --2- 172.21.15.16:0/4141058895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440cac30 0x7f85440cb0b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8544064640 tx=0x7f853c009840 comp rx=0 tx=0).stop 2024-08-17T02:45:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.592+0000 7f8549600640 1 -- 172.21.15.16:0/4141058895 shutdown_connections 2024-08-17T02:45:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.592+0000 7f8549600640 1 --2- 172.21.15.16:0/4141058895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440cac30 0x7f85440cb0b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.592+0000 7f8549600640 1 --2- 172.21.15.16:0/4141058895 >> v2:172.21.15.142:3300/0 conn(0x7f85440d08d0 0x7f85441590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.592+0000 7f8549600640 1 --2- 172.21.15.16:0/4141058895 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8544158ce0 0x7f85440c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.592+0000 7f8549600640 1 -- 172.21.15.16:0/4141058895 >> 172.21.15.16:0/4141058895 conn(0x7f85440cfdc0 msgr2=0x7f85440d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:16.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.592+0000 7f8549600640 1 -- 172.21.15.16:0/4141058895 shutdown_connections 2024-08-17T02:45:16.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.592+0000 7f8549600640 1 -- 172.21.15.16:0/4141058895 wait complete. 2024-08-17T02:45:16.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.594+0000 7f8549600640 1 Processor -- start 2024-08-17T02:45:16.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.594+0000 7f8549600640 1 -- start start 2024-08-17T02:45:16.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8549600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85440cac30 0x7f85440c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8549600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440d08d0 0x7f85440c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8549600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8544158ce0 0x7f854414c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8549600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85440ca010 con 0x7f85440d08d0 2024-08-17T02:45:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8549600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85441502d0 con 0x7f85440cac30 2024-08-17T02:45:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8549600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85440c9400 con 0x7f8544158ce0 2024-08-17T02:45:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8542000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440d08d0 0x7f85440c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8542000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440d08d0 0x7f85440c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32928/0 (socket says 172.21.15.16:32928) 2024-08-17T02:45:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8542000640 1 -- 172.21.15.16:0/1208307413 learned_addr learned my addr 172.21.15.16:0/1208307413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8542a00640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85440cac30 0x7f85440c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8543400640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8544158ce0 0x7f854414c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8542000640 1 -- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8544158ce0 msgr2=0x7f854414c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8542000640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8544158ce0 0x7f854414c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8542000640 1 -- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85440cac30 msgr2=0x7f85440c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8542000640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85440cac30 0x7f85440c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8542000640 1 -- 172.21.15.16:0/1208307413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c009ae0 con 0x7f85440d08d0 2024-08-17T02:45:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.595+0000 7f8542a00640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85440cac30 0x7f85440c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.596+0000 7f8542000640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440d08d0 0x7f85440c7e70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f853000ea20 tx=0x7f85300090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.596+0000 7f853a000640 1 -- 172.21.15.16:0/1208307413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853000ca10 con 0x7f85440d08d0 2024-08-17T02:45:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.596+0000 7f853a000640 1 -- 172.21.15.16:0/1208307413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853000cb70 con 0x7f85440d08d0 2024-08-17T02:45:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.596+0000 7f853a000640 1 -- 172.21.15.16:0/1208307413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530013470 con 0x7f85440d08d0 2024-08-17T02:45:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.596+0000 7f8549600640 1 -- 172.21.15.16:0/1208307413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85440c96e0 con 0x7f85440d08d0 2024-08-17T02:45:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.596+0000 7f8549600640 1 -- 172.21.15.16:0/1208307413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85440c9ca0 con 0x7f85440d08d0 2024-08-17T02:45:16.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.597+0000 7f8549600640 1 -- 172.21.15.16:0/1208307413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8544150440 con 0x7f85440d08d0 2024-08-17T02:45:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.598+0000 7f853a000640 1 -- 172.21.15.16:0/1208307413 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 83364+0+0 (secure 0 0 0) 0x7f8530013860 con 0x7f85440d08d0 2024-08-17T02:45:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.599+0000 7f853a000640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f85240650c0 0x7f8524067570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.599+0000 7f853a000640 1 -- 172.21.15.16:0/1208307413 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f85300824e0 con 0x7f85440d08d0 2024-08-17T02:45:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.599+0000 7f8542a00640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f85240650c0 0x7f8524067570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.600+0000 7f8542a00640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f85240650c0 0x7f8524067570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8534009c50 tx=0x7f8534008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.607+0000 7f853a000640 1 -- 172.21.15.16:0/1208307413 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8544150440 con 0x7f85440d08d0 2024-08-17T02:45:16.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.796+0000 7f8549600640 1 -- 172.21.15.16:0/1208307413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8544150440 con 0x7f85440d08d0 2024-08-17T02:45:16.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.796+0000 7f853a000640 1 -- 172.21.15.16:0/1208307413 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12140 (secure 0 0 0) 0x7f8544150440 con 0x7f85440d08d0 2024-08-17T02:45:16.797 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:16.797 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:15.992864+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.799+0000 7f8549600640 1 -- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f85240650c0 msgr2=0x7f8524067570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.799+0000 7f8549600640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f85240650c0 0x7f8524067570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8534009c50 tx=0x7f8534008040 comp rx=0 tx=0).stop 2024-08-17T02:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.799+0000 7f8549600640 1 -- 172.21.15.16:0/1208307413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440d08d0 msgr2=0x7f85440c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.799+0000 7f8549600640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440d08d0 0x7f85440c7e70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f853000ea20 tx=0x7f85300090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.799+0000 7f8549600640 1 -- 172.21.15.16:0/1208307413 shutdown_connections 2024-08-17T02:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.799+0000 7f8549600640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f85240650c0 0x7f8524067570 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.799+0000 7f8549600640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8544158ce0 0x7f854414c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.799+0000 7f8549600640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85440d08d0 0x7f85440c7e70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.799+0000 7f8549600640 1 --2- 172.21.15.16:0/1208307413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85440cac30 0x7f85440c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.799+0000 7f8549600640 1 -- 172.21.15.16:0/1208307413 >> 172.21.15.16:0/1208307413 conn(0x7f85440cfdc0 msgr2=0x7f85440c5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.799+0000 7f8549600640 1 -- 172.21.15.16:0/1208307413 shutdown_connections 2024-08-17T02:45:16.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.799+0000 7f8549600640 1 -- 172.21.15.16:0/1208307413 wait complete. 2024-08-17T02:45:16.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:45:14.291903+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-17T02:45:16.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:45:16.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.968+0000 7f274c400640 1 Processor -- start 2024-08-17T02:45:16.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.968+0000 7f274c400640 1 -- start start 2024-08-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.969+0000 7f274c400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27440b0a80 0x7f27440b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.970+0000 7f274c400640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2744157070 con 0x7f2744157e10 2024-08-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.970+0000 7f274c400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274415b410 con 0x7f27440adec0 2024-08-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.970+0000 7f274c400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27440b1390 con 0x7f27440b0a80 2024-08-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.970+0000 7f2749a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27440b0a80 0x7f27440b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.970+0000 7f2749000640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2744157e10 0x7f27440ad7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59688/0 (socket says 172.21.15.16:59688) 2024-08-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.970+0000 7f2749000640 1 -- 172.21.15.16:0/858806731 learned_addr learned my addr 172.21.15.16:0/858806731 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.970+0000 7f2749a00640 1 -- 172.21.15.16:0/858806731 >> v1:172.21.15.142:6789/0 conn(0x7f27440adec0 legacy=0x7f274415a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.970+0000 7f2749a00640 1 -- 172.21.15.16:0/858806731 >> v1:172.21.15.16:6789/0 conn(0x7f2744157e10 legacy=0x7f27440ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.970+0000 7f2749a00640 1 -- 172.21.15.16:0/858806731 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2744157330 con 0x7f27440b0a80 2024-08-17T02:45:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.970+0000 7f2743e00640 1 -- 172.21.15.16:0/858806731 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202347918 0 0) 0x7f2744157070 con 0x7f2744157e10 2024-08-17T02:45:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.971+0000 7f2749a00640 1 --2- 172.21.15.16:0/858806731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27440b0a80 0x7f27440b0e50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2738009850 tx=0x7f27380090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a18a78208fd2b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.972+0000 7f2743e00640 1 -- 172.21.15.16:0/858806731 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273802ad90 con 0x7f27440b0a80 2024-08-17T02:45:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.972+0000 7f2743e00640 1 -- 172.21.15.16:0/858806731 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2738032440 con 0x7f27440b0a80 2024-08-17T02:45:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.972+0000 7f2743e00640 1 -- 172.21.15.16:0/858806731 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27380327e0 con 0x7f27440b0a80 2024-08-17T02:45:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.972+0000 7f274c400640 1 -- 172.21.15.16:0/858806731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27440b0a80 msgr2=0x7f27440b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.972+0000 7f274c400640 1 --2- 172.21.15.16:0/858806731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27440b0a80 0x7f27440b0e50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2738009850 tx=0x7f27380090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.972+0000 7f274c400640 1 -- 172.21.15.16:0/858806731 shutdown_connections 2024-08-17T02:45:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.972+0000 7f274c400640 1 --2- 172.21.15.16:0/858806731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27440b0a80 0x7f27440b0e50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.972+0000 7f274c400640 1 -- 172.21.15.16:0/858806731 >> 172.21.15.16:0/858806731 conn(0x7f2744067750 msgr2=0x7f27440cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:16.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.973+0000 7f274c400640 1 -- 172.21.15.16:0/858806731 shutdown_connections 2024-08-17T02:45:16.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.973+0000 7f274c400640 1 -- 172.21.15.16:0/858806731 wait complete. 2024-08-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.974+0000 7f274c400640 1 Processor -- start 2024-08-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.974+0000 7f274c400640 1 -- start start 2024-08-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.975+0000 7f274c400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27440adec0 0x7f274414d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.975+0000 7f274c400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27440b0a80 0x7f274415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.975+0000 7f274c400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2744157e10 0x7f274415c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.975+0000 7f274c400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274414e700 con 0x7f2744157e10 2024-08-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.975+0000 7f274c400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274414ecc0 con 0x7f27440adec0 2024-08-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.975+0000 7f274c400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274414e4a0 con 0x7f27440b0a80 2024-08-17T02:45:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.975+0000 7f2749a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27440adec0 0x7f274414d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.975+0000 7f2749a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27440adec0 0x7f274414d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:38094/0 (socket says 172.21.15.16:38094) 2024-08-17T02:45:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.975+0000 7f274a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2744157e10 0x7f274415c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.975+0000 7f2749a00640 1 -- 172.21.15.16:0/2462919583 learned_addr learned my addr 172.21.15.16:0/2462919583 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.976+0000 7f2749000640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27440b0a80 0x7f274415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.976+0000 7f2749a00640 1 -- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27440b0a80 msgr2=0x7f274415c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.976+0000 7f2749a00640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27440b0a80 0x7f274415c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.976+0000 7f2749a00640 1 -- 172.21.15.16:0/2462919583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2744157e10 msgr2=0x7f274415c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.976+0000 7f2749a00640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2744157e10 0x7f274415c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:16.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.976+0000 7f2749a00640 1 -- 172.21.15.16:0/2462919583 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2738009390 con 0x7f27440adec0 2024-08-17T02:45:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.976+0000 7f2749000640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27440b0a80 0x7f274415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.976+0000 7f274a400640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2744157e10 0x7f274415c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.976+0000 7f2749a00640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27440adec0 0x7f274414d3d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2738009850 tx=0x7f2738032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.977+0000 7f2740c00640 1 -- 172.21.15.16:0/2462919583 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738031b30 con 0x7f27440adec0 2024-08-17T02:45:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.977+0000 7f2740c00640 1 -- 172.21.15.16:0/2462919583 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2738031cd0 con 0x7f27440adec0 2024-08-17T02:45:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.977+0000 7f2740c00640 1 -- 172.21.15.16:0/2462919583 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273803d670 con 0x7f27440adec0 2024-08-17T02:45:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.977+0000 7f274c400640 1 -- 172.21.15.16:0/2462919583 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274415cd80 con 0x7f27440adec0 2024-08-17T02:45:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.977+0000 7f274c400640 1 -- 172.21.15.16:0/2462919583 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274414e950 con 0x7f27440adec0 2024-08-17T02:45:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.978+0000 7f274c400640 1 -- 172.21.15.16:0/2462919583 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2744064db0 con 0x7f27440adec0 2024-08-17T02:45:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.979+0000 7f2740c00640 1 -- 172.21.15.16:0/2462919583 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 4) v1 ==== 83364+0+0 (secure 0 0 0) 0x7f2738039040 con 0x7f27440adec0 2024-08-17T02:45:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.980+0000 7f2740c00640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f27240650c0 0x7f2724067570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.981+0000 7f2740c00640 1 -- 172.21.15.16:0/2462919583 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f27380a1cc0 con 0x7f27440adec0 2024-08-17T02:45:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.981+0000 7f2749000640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f27240650c0 0x7f2724067570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.982+0000 7f2749000640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f27240650c0 0x7f2724067570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f27440b1870 tx=0x7f2730009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:16.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:16.985+0000 7f2740c00640 1 -- 172.21.15.16:0/2462919583 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f27380701c0 con 0x7f27440adec0 2024-08-17T02:45:17.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.005+0000 7f2740c00640 1 -- 172.21.15.16:0/2462919583 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f2738070590 con 0x7f27440adec0 2024-08-17T02:45:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.174+0000 7f274c400640 1 -- 172.21.15.16:0/2462919583 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2744159d60 con 0x7f27440adec0 2024-08-17T02:45:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.175+0000 7f2740c00640 1 -- 172.21.15.16:0/2462919583 <== mon.1 v2:172.21.15.142:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2738087e50 con 0x7f27440adec0 2024-08-17T02:45:17.176 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-17T02:45:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.177+0000 7f274c400640 1 -- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f27240650c0 msgr2=0x7f2724067570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.177+0000 7f274c400640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f27240650c0 0x7f2724067570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f27440b1870 tx=0x7f2730009040 comp rx=0 tx=0).stop 2024-08-17T02:45:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.177+0000 7f274c400640 1 -- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27440adec0 msgr2=0x7f274414d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.177+0000 7f274c400640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27440adec0 0x7f274414d3d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2738009850 tx=0x7f2738032000 comp rx=0 tx=0).stop 2024-08-17T02:45:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.177+0000 7f274c400640 1 -- 172.21.15.16:0/2462919583 shutdown_connections 2024-08-17T02:45:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.177+0000 7f274c400640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f27240650c0 0x7f2724067570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.177+0000 7f274c400640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2744157e10 0x7f274415c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.177+0000 7f274c400640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f27440b0a80 0x7f274415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.177+0000 7f274c400640 1 --2- 172.21.15.16:0/2462919583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27440adec0 0x7f274414d3d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.177+0000 7f274c400640 1 -- 172.21.15.16:0/2462919583 >> 172.21.15.16:0/2462919583 conn(0x7f2744067750 msgr2=0x7f27440cb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.177+0000 7f274c400640 1 -- 172.21.15.16:0/2462919583 shutdown_connections 2024-08-17T02:45:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.177+0000 7f274c400640 1 -- 172.21.15.16:0/2462919583 wait complete. 2024-08-17T02:45:17.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-08-17T02:45:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.343+0000 7fa96e000640 1 Processor -- start 2024-08-17T02:45:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.343+0000 7fa96e000640 1 -- start start 2024-08-17T02:45:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.344+0000 7fa96e000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9680aa8f0 0x7fa96806be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.344+0000 7fa96e000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa96815c600 0x7fa968066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.344+0000 7fa96e000640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96806b9d0 con 0x7fa96815c230 2024-08-17T02:45:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.344+0000 7fa96e000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96806ebf0 con 0x7fa9680aa8f0 2024-08-17T02:45:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.344+0000 7fa96e000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9680aad80 con 0x7fa96815c600 2024-08-17T02:45:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.344+0000 7fa966a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9680aa8f0 0x7fa96806be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.344+0000 7fa967400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa96815c230 0x7fa96806e4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59700/0 (socket says 172.21.15.16:59700) 2024-08-17T02:45:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.345+0000 7fa967400640 1 -- 172.21.15.16:0/1662275055 learned_addr learned my addr 172.21.15.16:0/1662275055 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.345+0000 7fa967e00640 1 --2- 172.21.15.16:0/1662275055 >> v2:172.21.15.142:3300/0 conn(0x7fa96815c600 0x7fa968066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.345+0000 7fa966a00640 1 -- 172.21.15.16:0/1662275055 >> v2:172.21.15.142:3300/0 conn(0x7fa96815c600 msgr2=0x7fa968066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.345+0000 7fa966a00640 1 --2- 172.21.15.16:0/1662275055 >> v2:172.21.15.142:3300/0 conn(0x7fa96815c600 0x7fa968066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.345+0000 7fa966a00640 1 -- 172.21.15.16:0/1662275055 >> v1:172.21.15.16:6789/0 conn(0x7fa96815c230 legacy=0x7fa96806e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.345+0000 7fa966a00640 1 -- 172.21.15.16:0/1662275055 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9680ab060 con 0x7fa9680aa8f0 2024-08-17T02:45:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.345+0000 7fa966000640 1 -- 172.21.15.16:0/1662275055 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868498298 0 0) 0x7fa96806b9d0 con 0x7fa96815c230 2024-08-17T02:45:17.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.346+0000 7fa966a00640 1 --2- 172.21.15.16:0/1662275055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9680aa8f0 0x7fa96806be60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa96806d6f0 tx=0x7fa9540090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47691e476f17390 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:17.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.346+0000 7fa966000640 1 -- 172.21.15.16:0/1662275055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95402adb0 con 0x7fa9680aa8f0 2024-08-17T02:45:17.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.346+0000 7fa966000640 1 -- 172.21.15.16:0/1662275055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa954032440 con 0x7fa9680aa8f0 2024-08-17T02:45:17.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.346+0000 7fa966000640 1 -- 172.21.15.16:0/1662275055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9540327e0 con 0x7fa9680aa8f0 2024-08-17T02:45:17.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.347+0000 7fa96e000640 1 -- 172.21.15.16:0/1662275055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9680aa8f0 msgr2=0x7fa96806be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.347+0000 7fa96e000640 1 --2- 172.21.15.16:0/1662275055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9680aa8f0 0x7fa96806be60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa96806d6f0 tx=0x7fa9540090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.347+0000 7fa96e000640 1 -- 172.21.15.16:0/1662275055 shutdown_connections 2024-08-17T02:45:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.347+0000 7fa96e000640 1 --2- 172.21.15.16:0/1662275055 >> v2:172.21.15.142:3300/0 conn(0x7fa96815c600 0x7fa968066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.347+0000 7fa96e000640 1 --2- 172.21.15.16:0/1662275055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9680aa8f0 0x7fa96806be60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.347+0000 7fa96e000640 1 -- 172.21.15.16:0/1662275055 >> 172.21.15.16:0/1662275055 conn(0x7fa96806d8f0 msgr2=0x7fa96806dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.347+0000 7fa96e000640 1 -- 172.21.15.16:0/1662275055 shutdown_connections 2024-08-17T02:45:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.347+0000 7fa96e000640 1 -- 172.21.15.16:0/1662275055 wait complete. 2024-08-17T02:45:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.348+0000 7fa96e000640 1 Processor -- start 2024-08-17T02:45:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.348+0000 7fa96e000640 1 -- start start 2024-08-17T02:45:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.349+0000 7fa96e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680aa8f0 0x7fa9681689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.349+0000 7fa96e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96815c230 0x7fa96815af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.349+0000 7fa96e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96815c600 0x7fa96815b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.349+0000 7fa96e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9680c75f0 con 0x7fa96815c230 2024-08-17T02:45:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.349+0000 7fa96e000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa968169c10 con 0x7fa96815c600 2024-08-17T02:45:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.349+0000 7fa96e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa968168fe0 con 0x7fa9680aa8f0 2024-08-17T02:45:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.350+0000 7fa966a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96815c230 0x7fa96815af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.350+0000 7fa967400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680aa8f0 0x7fa9681689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.350+0000 7fa966a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96815c230 0x7fa96815af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32974/0 (socket says 172.21.15.16:32974) 2024-08-17T02:45:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.350+0000 7fa966a00640 1 -- 172.21.15.16:0/4196158054 learned_addr learned my addr 172.21.15.16:0/4196158054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.350+0000 7fa967e00640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96815c600 0x7fa96815b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.350+0000 7fa966a00640 1 -- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680aa8f0 msgr2=0x7fa9681689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.350+0000 7fa966a00640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680aa8f0 0x7fa9681689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.350+0000 7fa966a00640 1 -- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96815c600 msgr2=0x7fa96815b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.350+0000 7fa966a00640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96815c600 0x7fa96815b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.350+0000 7fa966a00640 1 -- 172.21.15.16:0/4196158054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa954009390 con 0x7fa96815c230 2024-08-17T02:45:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.350+0000 7fa966a00640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96815c230 0x7fa96815af40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa954035040 tx=0x7fa954032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.351+0000 7fa95ea00640 1 -- 172.21.15.16:0/4196158054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954000d10 con 0x7fa96815c230 2024-08-17T02:45:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.351+0000 7fa95ea00640 1 -- 172.21.15.16:0/4196158054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa954031e00 con 0x7fa96815c230 2024-08-17T02:45:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.351+0000 7fa96e000640 1 -- 172.21.15.16:0/4196158054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9681692c0 con 0x7fa96815c230 2024-08-17T02:45:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.351+0000 7fa95ea00640 1 -- 172.21.15.16:0/4196158054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95403d670 con 0x7fa96815c230 2024-08-17T02:45:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.351+0000 7fa96e000640 1 -- 172.21.15.16:0/4196158054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa968169880 con 0x7fa96815c230 2024-08-17T02:45:17.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.352+0000 7fa96e000640 1 -- 172.21.15.16:0/4196158054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa968064db0 con 0x7fa96815c230 2024-08-17T02:45:17.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.353+0000 7fa95ea00640 1 -- 172.21.15.16:0/4196158054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fa954039040 con 0x7fa96815c230 2024-08-17T02:45:17.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.354+0000 7fa95ea00640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa948065150 0x7fa948067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:17.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.354+0000 7fa95ea00640 1 -- 172.21.15.16:0/4196158054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa9540a2230 con 0x7fa96815c230 2024-08-17T02:45:17.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.354+0000 7fa967400640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa948065150 0x7fa948067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:17.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.355+0000 7fa967400640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa948065150 0x7fa948067600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9580011e0 tx=0x7fa958007ca0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:17.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.359+0000 7fa95ea00640 1 -- 172.21.15.16:0/4196158054 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa9540706f0 con 0x7fa96815c230 2024-08-17T02:45:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.545+0000 7fa96e000640 1 -- 172.21.15.16:0/4196158054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa96806e100 con 0x7fa96815c230 2024-08-17T02:45:17.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.587+0000 7fa95ea00640 1 -- 172.21.15.16:0/4196158054 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa954073d10 con 0x7fa96815c230 2024-08-17T02:45:17.589 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:17.589 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-08-17T02:45:17.589 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:45:17.589 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-08-17T02:45:17.589 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-08-17T02:45:17.589 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:45:17.589 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-08-17T02:45:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.589+0000 7fa96e000640 1 -- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa948065150 msgr2=0x7fa948067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.589+0000 7fa96e000640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa948065150 0x7fa948067600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9580011e0 tx=0x7fa958007ca0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.589+0000 7fa96e000640 1 -- 172.21.15.16:0/4196158054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96815c230 msgr2=0x7fa96815af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.589+0000 7fa96e000640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96815c230 0x7fa96815af40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa954035040 tx=0x7fa954032000 comp rx=0 tx=0).stop 2024-08-17T02:45:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.589+0000 7fa96e000640 1 -- 172.21.15.16:0/4196158054 shutdown_connections 2024-08-17T02:45:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.589+0000 7fa96e000640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa948065150 0x7fa948067600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.589+0000 7fa96e000640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa96815c600 0x7fa96815b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.589+0000 7fa96e000640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96815c230 0x7fa96815af40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.589+0000 7fa96e000640 1 --2- 172.21.15.16:0/4196158054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa9680aa8f0 0x7fa9681689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.589+0000 7fa96e000640 1 -- 172.21.15.16:0/4196158054 >> 172.21.15.16:0/4196158054 conn(0x7fa96806d8f0 msgr2=0x7fa968066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.590+0000 7fa96e000640 1 -- 172.21.15.16:0/4196158054 shutdown_connections 2024-08-17T02:45:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.590+0000 7fa96e000640 1 -- 172.21.15.16:0/4196158054 wait complete. 2024-08-17T02:45:17.605 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-17T02:45:17.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-17T02:45:17.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.760+0000 7f23f8600640 1 Processor -- start 2024-08-17T02:45:17.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.761+0000 7f23f8600640 1 -- start start 2024-08-17T02:45:17.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.762+0000 7f23f8600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f23f00b0a80 0x7f23f00b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:17.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.762+0000 7f23f8600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23f0157e10 0x7f23f00ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:17.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.762+0000 7f23f8600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f00ae110 con 0x7f23f00adcf0 2024-08-17T02:45:17.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.762+0000 7f23f8600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23f00b1390 con 0x7f23f0157e10 2024-08-17T02:45:17.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.762+0000 7f23f8600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23f0157070 con 0x7f23f00b0a80 2024-08-17T02:45:17.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.762+0000 7f23f5200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23f0157e10 0x7f23f00ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:17.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.762+0000 7f23f5c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f23f00b0a80 0x7f23f00b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:17.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.762+0000 7f23f6600640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f23f00adcf0 0x7f23f015a6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59702/0 (socket says 172.21.15.16:59702) 2024-08-17T02:45:17.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.762+0000 7f23f6600640 1 -- 172.21.15.16:0/2991465765 learned_addr learned my addr 172.21.15.16:0/2991465765 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:17.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.763+0000 7f23f5c00640 1 -- 172.21.15.16:0/2991465765 >> v2:172.21.15.16:3300/0 conn(0x7f23f0157e10 msgr2=0x7f23f00ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.763+0000 7f23f5c00640 1 --2- 172.21.15.16:0/2991465765 >> v2:172.21.15.16:3300/0 conn(0x7f23f0157e10 0x7f23f00ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.763+0000 7f23f5c00640 1 -- 172.21.15.16:0/2991465765 >> v1:172.21.15.16:6789/0 conn(0x7f23f00adcf0 legacy=0x7f23f015a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.763+0000 7f23f5c00640 1 -- 172.21.15.16:0/2991465765 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f015b410 con 0x7f23f00b0a80 2024-08-17T02:45:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.764+0000 7f23f5c00640 1 --2- 172.21.15.16:0/2991465765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f00b0a80 0x7f23f00b0e50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f23f0068410 tx=0x7f23e40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70476bab00bf5bd8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:17.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.764+0000 7f23efe00640 1 -- 172.21.15.16:0/2991465765 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e402ad20 con 0x7f23f00b0a80 2024-08-17T02:45:17.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.764+0000 7f23efe00640 1 -- 172.21.15.16:0/2991465765 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e402ae80 con 0x7f23f00b0a80 2024-08-17T02:45:17.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.764+0000 7f23efe00640 1 -- 172.21.15.16:0/2991465765 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4032670 con 0x7f23f00b0a80 2024-08-17T02:45:17.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.764+0000 7f23f8600640 1 -- 172.21.15.16:0/2991465765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f00b0a80 msgr2=0x7f23f00b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:17.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.764+0000 7f23f8600640 1 --2- 172.21.15.16:0/2991465765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f00b0a80 0x7f23f00b0e50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f23f0068410 tx=0x7f23e40090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.765+0000 7f23f8600640 1 -- 172.21.15.16:0/2991465765 shutdown_connections 2024-08-17T02:45:17.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.765+0000 7f23f8600640 1 --2- 172.21.15.16:0/2991465765 >> v2:172.21.15.16:3300/0 conn(0x7f23f0157e10 0x7f23f00ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.765+0000 7f23f8600640 1 --2- 172.21.15.16:0/2991465765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f00b0a80 0x7f23f00b0e50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.765+0000 7f23f8600640 1 -- 172.21.15.16:0/2991465765 >> 172.21.15.16:0/2991465765 conn(0x7f23f0067750 msgr2=0x7f23f00cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:17.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.765+0000 7f23f8600640 1 -- 172.21.15.16:0/2991465765 shutdown_connections 2024-08-17T02:45:17.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.765+0000 7f23f8600640 1 -- 172.21.15.16:0/2991465765 wait complete. 2024-08-17T02:45:17.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.766+0000 7f23f8600640 1 Processor -- start 2024-08-17T02:45:17.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.766+0000 7f23f8600640 1 -- start start 2024-08-17T02:45:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.767+0000 7f23f8600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f00adcf0 0x7f23f01542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.767+0000 7f23f8600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f00b0a80 0x7f23f01521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.767+0000 7f23f8600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f0157e10 0x7f23f014d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.767+0000 7f23f8600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f0152e20 con 0x7f23f0157e10 2024-08-17T02:45:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.767+0000 7f23f8600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f0152b40 con 0x7f23f00adcf0 2024-08-17T02:45:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.767+0000 7f23f8600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f014f4b0 con 0x7f23f00b0a80 2024-08-17T02:45:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.767+0000 7f23f6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f0157e10 0x7f23f014d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.767+0000 7f23f6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f0157e10 0x7f23f014d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32994/0 (socket says 172.21.15.16:32994) 2024-08-17T02:45:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.767+0000 7f23f6600640 1 -- 172.21.15.16:0/1903911788 learned_addr learned my addr 172.21.15.16:0/1903911788 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.767+0000 7f23f5c00640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f00adcf0 0x7f23f01542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:17.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.768+0000 7f23f5200640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f00b0a80 0x7f23f01521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:17.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.768+0000 7f23f6600640 1 -- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f00b0a80 msgr2=0x7f23f01521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:17.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.768+0000 7f23f6600640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f00b0a80 0x7f23f01521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.768+0000 7f23f6600640 1 -- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f00adcf0 msgr2=0x7f23f01542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:17.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.768+0000 7f23f6600640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f00adcf0 0x7f23f01542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:17.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.768+0000 7f23f6600640 1 -- 172.21.15.16:0/1903911788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e4009390 con 0x7f23f0157e10 2024-08-17T02:45:17.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.768+0000 7f23f6600640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f0157e10 0x7f23f014d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f23e800cbc0 tx=0x7f23e80019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:17.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.768+0000 7f23ecc00640 1 -- 172.21.15.16:0/1903911788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e800a640 con 0x7f23f0157e10 2024-08-17T02:45:17.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.768+0000 7f23ecc00640 1 -- 172.21.15.16:0/1903911788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e800a7a0 con 0x7f23f0157e10 2024-08-17T02:45:17.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.768+0000 7f23f8600640 1 -- 172.21.15.16:0/1903911788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f014f790 con 0x7f23f0157e10 2024-08-17T02:45:17.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.768+0000 7f23ecc00640 1 -- 172.21.15.16:0/1903911788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e800ab20 con 0x7f23f0157e10 2024-08-17T02:45:17.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.769+0000 7f23f8600640 1 -- 172.21.15.16:0/1903911788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f0152730 con 0x7f23f0157e10 2024-08-17T02:45:17.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.770+0000 7f23f8600640 1 -- 172.21.15.16:0/1903911788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f014f310 con 0x7f23f0157e10 2024-08-17T02:45:17.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.771+0000 7f23ecc00640 1 -- 172.21.15.16:0/1903911788 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f23e8007300 con 0x7f23f0157e10 2024-08-17T02:45:17.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.772+0000 7f23ecc00640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f23d0065220 0x7f23d00676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:17.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.772+0000 7f23ecc00640 1 -- 172.21.15.16:0/1903911788 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f23e8082ce0 con 0x7f23f0157e10 2024-08-17T02:45:17.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.772+0000 7f23f5c00640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f23d0065220 0x7f23d00676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:17.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.773+0000 7f23f5c00640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f23d0065220 0x7f23d00676d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23e4034000 tx=0x7f23e4032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:17.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.775+0000 7f23ecc00640 1 -- 172.21.15.16:0/1903911788 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f23e80510c0 con 0x7f23f0157e10 2024-08-17T02:45:17.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:17.964+0000 7f23f8600640 1 -- 172.21.15.16:0/1903911788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f23f014f310 con 0x7f23f0157e10 2024-08-17T02:45:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.012+0000 7f23ecc00640 1 -- 172.21.15.16:0/1903911788 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f23f014f310 con 0x7f23f0157e10 2024-08-17T02:45:18.013 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-08-17T02:45:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.014+0000 7f23f8600640 1 -- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f23d0065220 msgr2=0x7f23d00676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.014+0000 7f23f8600640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f23d0065220 0x7f23d00676d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23e4034000 tx=0x7f23e4032000 comp rx=0 tx=0).stop 2024-08-17T02:45:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.014+0000 7f23f8600640 1 -- 172.21.15.16:0/1903911788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f0157e10 msgr2=0x7f23f014d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.014+0000 7f23f8600640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f0157e10 0x7f23f014d390 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f23e800cbc0 tx=0x7f23e80019c0 comp rx=0 tx=0).stop 2024-08-17T02:45:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.015+0000 7f23f8600640 1 -- 172.21.15.16:0/1903911788 shutdown_connections 2024-08-17T02:45:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.015+0000 7f23f8600640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f23d0065220 0x7f23d00676d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.015+0000 7f23f8600640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f0157e10 0x7f23f014d390 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:18.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.015+0000 7f23f8600640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f23f00b0a80 0x7f23f01521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.015+0000 7f23f8600640 1 --2- 172.21.15.16:0/1903911788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23f00adcf0 0x7f23f01542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.015+0000 7f23f8600640 1 -- 172.21.15.16:0/1903911788 >> 172.21.15.16:0/1903911788 conn(0x7f23f0067750 msgr2=0x7f23f015a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.015+0000 7f23f8600640 1 -- 172.21.15.16:0/1903911788 shutdown_connections 2024-08-17T02:45:18.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.015+0000 7f23f8600640 1 -- 172.21.15.16:0/1903911788 wait complete. 2024-08-17T02:45:18.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-17T02:45:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.192+0000 7fce1ae00640 1 Processor -- start 2024-08-17T02:45:18.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.192+0000 7fce1ae00640 1 -- start start 2024-08-17T02:45:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.193+0000 7fce1ae00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce140cf290 0x7fce14159180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.193+0000 7fce1ae00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce140cb4c0 con 0x7fce140caa40 2024-08-17T02:45:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.193+0000 7fce1ae00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce140caec0 con 0x7fce14158db0 2024-08-17T02:45:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.193+0000 7fce1ae00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce140cf770 con 0x7fce140cf290 2024-08-17T02:45:18.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.194+0000 7fce0be00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce140cf290 0x7fce14159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.194+0000 7fce18e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fce140caa40 0x7fce140c6da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59710/0 (socket says 172.21.15.16:59710) 2024-08-17T02:45:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.194+0000 7fce18e00640 1 -- 172.21.15.16:0/1366181302 learned_addr learned my addr 172.21.15.16:0/1366181302 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.194+0000 7fce0be00640 1 -- 172.21.15.16:0/1366181302 >> v1:172.21.15.142:6789/0 conn(0x7fce14158db0 legacy=0x7fce140ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.194+0000 7fce0be00640 1 -- 172.21.15.16:0/1366181302 >> v1:172.21.15.16:6789/0 conn(0x7fce140caa40 legacy=0x7fce140c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.194+0000 7fce13400640 1 -- 172.21.15.16:0/1366181302 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640709212 0 0) 0x7fce140caec0 con 0x7fce14158db0 2024-08-17T02:45:18.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.194+0000 7fce0be00640 1 -- 172.21.15.16:0/1366181302 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140cfa50 con 0x7fce140cf290 2024-08-17T02:45:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.195+0000 7fce0be00640 1 --2- 172.21.15.16:0/1366181302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce140cf290 0x7fce14159180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009850 tx=0x7fcdfc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63154356059c26bd server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.195+0000 7fce13400640 1 -- 172.21.15.16:0/1366181302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc02ad10 con 0x7fce140cf290 2024-08-17T02:45:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.195+0000 7fce13400640 1 -- 172.21.15.16:0/1366181302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc02ae70 con 0x7fce140cf290 2024-08-17T02:45:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.195+0000 7fce13400640 1 -- 172.21.15.16:0/1366181302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc032650 con 0x7fce140cf290 2024-08-17T02:45:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.196+0000 7fce1ae00640 1 -- 172.21.15.16:0/1366181302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce140cf290 msgr2=0x7fce14159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:18.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.196+0000 7fce1ae00640 1 --2- 172.21.15.16:0/1366181302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce140cf290 0x7fce14159180 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009850 tx=0x7fcdfc0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.196+0000 7fce1ae00640 1 -- 172.21.15.16:0/1366181302 shutdown_connections 2024-08-17T02:45:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.196+0000 7fce1ae00640 1 --2- 172.21.15.16:0/1366181302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce140cf290 0x7fce14159180 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.196+0000 7fce1ae00640 1 -- 172.21.15.16:0/1366181302 >> 172.21.15.16:0/1366181302 conn(0x7fce140cfd90 msgr2=0x7fce140d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.196+0000 7fce1ae00640 1 -- 172.21.15.16:0/1366181302 shutdown_connections 2024-08-17T02:45:18.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.196+0000 7fce1ae00640 1 -- 172.21.15.16:0/1366181302 wait complete. 2024-08-17T02:45:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.197+0000 7fce1ae00640 1 Processor -- start 2024-08-17T02:45:18.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.198+0000 7fce1ae00640 1 -- start start 2024-08-17T02:45:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.198+0000 7fce1ae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce140caa40 0x7fce1409d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.198+0000 7fce1ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce140cf290 0x7fce1409b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.198+0000 7fce1ae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce14158db0 0x7fce140b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.198+0000 7fce1ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce140b45a0 con 0x7fce140cf290 2024-08-17T02:45:18.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.198+0000 7fce1ae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce14156b60 con 0x7fce14158db0 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.198+0000 7fce1ae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce140c3db0 con 0x7fce140caa40 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.199+0000 7fce0be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce140cf290 0x7fce1409b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.199+0000 7fce0be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce140cf290 0x7fce1409b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33022/0 (socket says 172.21.15.16:33022) 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.199+0000 7fce0be00640 1 -- 172.21.15.16:0/2831428781 learned_addr learned my addr 172.21.15.16:0/2831428781 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.199+0000 7fce18e00640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce14158db0 0x7fce140b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.199+0000 7fce0be00640 1 -- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce140caa40 msgr2=0x7fce1409d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.199+0000 7fce0be00640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce140caa40 0x7fce1409d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.199+0000 7fce0be00640 1 -- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce14158db0 msgr2=0x7fce140b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.199+0000 7fce0be00640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce14158db0 0x7fce140b4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.199+0000 7fce0be00640 1 -- 172.21.15.16:0/2831428781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc009390 con 0x7fce140cf290 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.199+0000 7fce0be00640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce140cf290 0x7fce1409b440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009850 tx=0x7fcdfc032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.200+0000 7fce0b400640 1 -- 172.21.15.16:0/2831428781 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc031880 con 0x7fce140cf290 2024-08-17T02:45:18.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.200+0000 7fce0b400640 1 -- 172.21.15.16:0/2831428781 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc031a00 con 0x7fce140cf290 2024-08-17T02:45:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.200+0000 7fce0b400640 1 -- 172.21.15.16:0/2831428781 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc031dc0 con 0x7fce140cf290 2024-08-17T02:45:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.200+0000 7fce1ae00640 1 -- 172.21.15.16:0/2831428781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce140c4090 con 0x7fce140cf290 2024-08-17T02:45:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.200+0000 7fce1ae00640 1 -- 172.21.15.16:0/2831428781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce140c4650 con 0x7fce140cf290 2024-08-17T02:45:18.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.201+0000 7fce1ae00640 1 -- 172.21.15.16:0/2831428781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce140c4870 con 0x7fce140cf290 2024-08-17T02:45:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.202+0000 7fce0b400640 1 -- 172.21.15.16:0/2831428781 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fcdfc044030 con 0x7fce140cf290 2024-08-17T02:45:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.203+0000 7fce0b400640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcdf0065150 0x7fcdf0067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.203+0000 7fce0b400640 1 -- 172.21.15.16:0/2831428781 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fcdfc042070 con 0x7fce140cf290 2024-08-17T02:45:18.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.203+0000 7fce13e00640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcdf0065150 0x7fcdf0067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:18.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.204+0000 7fce13e00640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcdf0065150 0x7fcdf0067600 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce04001640 tx=0x7fce04007ab0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:18.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.207+0000 7fce0b400640 1 -- 172.21.15.16:0/2831428781 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcdfc071040 con 0x7fce140cf290 2024-08-17T02:45:18.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:18.389+0000 7fce1ae00640 1 -- 172.21.15.16:0/2831428781 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fce140c3ef0 con 0x7fce140cf290 2024-08-17T02:45:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.018+0000 7fce0b400640 1 -- 172.21.15.16:0/2831428781 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fce140c3ef0 con 0x7fce140cf290 2024-08-17T02:45:19.020 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-08-17T02:45:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.025+0000 7fce1ae00640 1 -- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcdf0065150 msgr2=0x7fcdf0067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.025+0000 7fce1ae00640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcdf0065150 0x7fcdf0067600 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce04001640 tx=0x7fce04007ab0 comp rx=0 tx=0).stop 2024-08-17T02:45:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.025+0000 7fce1ae00640 1 -- 172.21.15.16:0/2831428781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce140cf290 msgr2=0x7fce1409b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.025+0000 7fce1ae00640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce140cf290 0x7fce1409b440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009850 tx=0x7fcdfc032000 comp rx=0 tx=0).stop 2024-08-17T02:45:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.025+0000 7fce1ae00640 1 -- 172.21.15.16:0/2831428781 shutdown_connections 2024-08-17T02:45:19.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.025+0000 7fce1ae00640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcdf0065150 0x7fcdf0067600 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce04001640 tx=0x7fce04007ab0 comp rx=0 tx=0).stop 2024-08-17T02:45:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.025+0000 7fce1ae00640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce14158db0 0x7fce140b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.025+0000 7fce1ae00640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce140cf290 0x7fce1409b440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.025+0000 7fce1ae00640 1 --2- 172.21.15.16:0/2831428781 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce140caa40 0x7fce1409d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.025+0000 7fce1ae00640 1 -- 172.21.15.16:0/2831428781 >> 172.21.15.16:0/2831428781 conn(0x7fce140cfd90 msgr2=0x7fce14158390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.026+0000 7fce1ae00640 1 -- 172.21.15.16:0/2831428781 shutdown_connections 2024-08-17T02:45:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.026+0000 7fce1ae00640 1 -- 172.21.15.16:0/2831428781 wait complete. 2024-08-17T02:45:19.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-17T02:45:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.210+0000 7f46c5200640 1 Processor -- start 2024-08-17T02:45:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.210+0000 7f46c5200640 1 -- start start 2024-08-17T02:45:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.211+0000 7f46c5200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46c00c7db0 0x7f46c00c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46c5200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c01598f0 con 0x7f46c0155a80 2024-08-17T02:45:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46c5200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c0159bd0 con 0x7f46c0155e50 2024-08-17T02:45:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46c5200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c0159eb0 con 0x7f46c00c7db0 2024-08-17T02:45:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46bde00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f46c0155a80 0x7f46c006dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59724/0 (socket says 172.21.15.16:59724) 2024-08-17T02:45:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46bde00640 1 -- 172.21.15.16:0/3232824005 learned_addr learned my addr 172.21.15.16:0/3232824005 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46bd400640 1 --2- 172.21.15.16:0/3232824005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46c00c7db0 0x7f46c00c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46bca00640 1 -- 172.21.15.16:0/3232824005 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656359743 0 0) 0x7f46c01598f0 con 0x7f46c0155a80 2024-08-17T02:45:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46bca00640 1 -- 172.21.15.16:0/3232824005 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46a40034e0 con 0x7f46c0155a80 2024-08-17T02:45:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46bca00640 1 -- 172.21.15.16:0/3232824005 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130075404 0 0) 0x7f46c0159bd0 con 0x7f46c0155e50 2024-08-17T02:45:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46bca00640 1 -- 172.21.15.16:0/3232824005 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46c01598f0 con 0x7f46c0155e50 2024-08-17T02:45:19.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46bd400640 1 -- 172.21.15.16:0/3232824005 >> v1:172.21.15.142:6789/0 conn(0x7f46c0155e50 legacy=0x7f46c00aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:19.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46bca00640 1 -- 172.21.15.16:0/3232824005 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3046665213 0 0) 0x7f46a40034e0 con 0x7f46c0155a80 2024-08-17T02:45:19.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46bd400640 1 -- 172.21.15.16:0/3232824005 >> v1:172.21.15.16:6789/0 conn(0x7f46c0155a80 legacy=0x7f46c006dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:19.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.212+0000 7f46bd400640 1 -- 172.21.15.16:0/3232824005 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c015a190 con 0x7f46c00c7db0 2024-08-17T02:45:19.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.214+0000 7f46bd400640 1 --2- 172.21.15.16:0/3232824005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46c00c7db0 0x7f46c00c8bd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009880 tx=0x7f46ac0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f84aafddfd9940fe server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:19.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.214+0000 7f46bca00640 1 -- 172.21.15.16:0/3232824005 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b0001470 con 0x7f46c00c7db0 2024-08-17T02:45:19.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.214+0000 7f46bca00640 1 -- 172.21.15.16:0/3232824005 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac02aac0 con 0x7f46c00c7db0 2024-08-17T02:45:19.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.214+0000 7f46bca00640 1 -- 172.21.15.16:0/3232824005 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac02ae60 con 0x7f46c00c7db0 2024-08-17T02:45:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.214+0000 7f46c5200640 1 -- 172.21.15.16:0/3232824005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46c00c7db0 msgr2=0x7f46c00c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.214+0000 7f46c5200640 1 --2- 172.21.15.16:0/3232824005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46c00c7db0 0x7f46c00c8bd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009880 tx=0x7f46ac0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.214+0000 7f46c5200640 1 -- 172.21.15.16:0/3232824005 shutdown_connections 2024-08-17T02:45:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.214+0000 7f46c5200640 1 --2- 172.21.15.16:0/3232824005 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46c00c7db0 0x7f46c00c8bd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.214+0000 7f46c5200640 1 -- 172.21.15.16:0/3232824005 >> 172.21.15.16:0/3232824005 conn(0x7f46c0067750 msgr2=0x7f46c00c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.215+0000 7f46c5200640 1 -- 172.21.15.16:0/3232824005 shutdown_connections 2024-08-17T02:45:19.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.215+0000 7f46c5200640 1 -- 172.21.15.16:0/3232824005 wait complete. 2024-08-17T02:45:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.216+0000 7f46c5200640 1 Processor -- start 2024-08-17T02:45:19.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.216+0000 7f46c5200640 1 -- start start 2024-08-17T02:45:19.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.217+0000 7f46c5200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c00c7db0 0x7f46c0158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:19.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.217+0000 7f46c5200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46c0155a80 0x7f46c0157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:19.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.217+0000 7f46c5200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46c0155e50 0x7f46c00adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:19.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.217+0000 7f46c5200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c0169440 con 0x7f46c00c7db0 2024-08-17T02:45:19.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.217+0000 7f46c5200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c0145640 con 0x7f46c0155e50 2024-08-17T02:45:19.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.217+0000 7f46c5200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c0145920 con 0x7f46c0155a80 2024-08-17T02:45:19.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.217+0000 7f46bde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c00c7db0 0x7f46c0158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:19.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.217+0000 7f46bde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c00c7db0 0x7f46c0158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33028/0 (socket says 172.21.15.16:33028) 2024-08-17T02:45:19.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.217+0000 7f46bde00640 1 -- 172.21.15.16:0/3502646220 learned_addr learned my addr 172.21.15.16:0/3502646220 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:19.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.217+0000 7f46bd400640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46c0155a80 0x7f46c0157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:19.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.217+0000 7f46be800640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46c0155e50 0x7f46c00adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:19.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46bde00640 1 -- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46c0155a80 msgr2=0x7f46c0157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:19.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46bde00640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46c0155a80 0x7f46c0157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:19.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46bde00640 1 -- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46c0155e50 msgr2=0x7f46c00adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:19.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46bde00640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46c0155e50 0x7f46c00adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:19.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46bde00640 1 -- 172.21.15.16:0/3502646220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac009390 con 0x7f46c00c7db0 2024-08-17T02:45:19.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46bd400640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46c0155a80 0x7f46c0157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:19.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46be800640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46c0155e50 0x7f46c00adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:19.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46bde00640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c00c7db0 0x7f46c0158ba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f46b000c960 tx=0x7f46b0001ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:19.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46b5600640 1 -- 172.21.15.16:0/3502646220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b0007360 con 0x7f46c00c7db0 2024-08-17T02:45:19.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46b5600640 1 -- 172.21.15.16:0/3502646220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b00074e0 con 0x7f46c00c7db0 2024-08-17T02:45:19.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46c5200640 1 -- 172.21.15.16:0/3502646220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c0145c00 con 0x7f46c00c7db0 2024-08-17T02:45:19.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46b5600640 1 -- 172.21.15.16:0/3502646220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b00049f0 con 0x7f46c00c7db0 2024-08-17T02:45:19.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.218+0000 7f46c5200640 1 -- 172.21.15.16:0/3502646220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c0145280 con 0x7f46c00c7db0 2024-08-17T02:45:19.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.219+0000 7f46c5200640 1 -- 172.21.15.16:0/3502646220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c0146100 con 0x7f46c00c7db0 2024-08-17T02:45:19.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.220+0000 7f46b5600640 1 -- 172.21.15.16:0/3502646220 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f46b000a640 con 0x7f46c00c7db0 2024-08-17T02:45:19.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.221+0000 7f46b5600640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f46a0065150 0x7f46a0067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:19.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.222+0000 7f46b5600640 1 -- 172.21.15.16:0/3502646220 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f46b0081d00 con 0x7f46c00c7db0 2024-08-17T02:45:19.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.222+0000 7f46bd400640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f46a0065150 0x7f46a0067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:19.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.222+0000 7f46bd400640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f46a0065150 0x7f46a0067600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f46ac034000 tx=0x7f46ac032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:19.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.226+0000 7f46b5600640 1 -- 172.21.15.16:0/3502646220 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f46b004fe00 con 0x7f46c00c7db0 2024-08-17T02:45:19.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:19.448+0000 7f46c5200640 1 -- 172.21.15.16:0/3502646220 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f46c0071050 con 0x7f46c00c7db0 2024-08-17T02:45:19.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:45:19.653 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:45:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.035+0000 7f46b5600640 1 -- 172.21.15.16:0/3502646220 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f46b0053420 con 0x7f46c00c7db0 2024-08-17T02:45:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-08-17T02:45:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.038+0000 7f46c5200640 1 -- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f46a0065150 msgr2=0x7f46a0067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.039+0000 7f46c5200640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f46a0065150 0x7f46a0067600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f46ac034000 tx=0x7f46ac032000 comp rx=0 tx=0).stop 2024-08-17T02:45:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.039+0000 7f46c5200640 1 -- 172.21.15.16:0/3502646220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c00c7db0 msgr2=0x7f46c0158ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.039+0000 7f46c5200640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c00c7db0 0x7f46c0158ba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f46b000c960 tx=0x7f46b0001ba0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.040+0000 7f46c5200640 1 -- 172.21.15.16:0/3502646220 shutdown_connections 2024-08-17T02:45:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.040+0000 7f46c5200640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f46a0065150 0x7f46a0067600 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.040+0000 7f46c5200640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46c0155e50 0x7f46c00adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.040+0000 7f46c5200640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f46c0155a80 0x7f46c0157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.040+0000 7f46c5200640 1 --2- 172.21.15.16:0/3502646220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46c00c7db0 0x7f46c0158ba0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.040+0000 7f46c5200640 1 -- 172.21.15.16:0/3502646220 >> 172.21.15.16:0/3502646220 conn(0x7f46c0067750 msgr2=0x7f46c01476d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.042+0000 7f46c5200640 1 -- 172.21.15.16:0/3502646220 shutdown_connections 2024-08-17T02:45:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.042+0000 7f46c5200640 1 -- 172.21.15.16:0/3502646220 wait complete. 2024-08-17T02:45:20.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:20.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.215+0000 7f0eca600640 1 Processor -- start 2024-08-17T02:45:20.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.215+0000 7f0eca600640 1 -- start start 2024-08-17T02:45:20.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.216+0000 7f0eca600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0ec406df00 0x7f0ec40adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.216+0000 7f0eca600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ec4155df0 0x7f0ec40ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.216+0000 7f0eca600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec4156670 con 0x7f0ec4155a20 2024-08-17T02:45:20.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.216+0000 7f0eca600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec40b0350 con 0x7f0ec4155df0 2024-08-17T02:45:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.216+0000 7f0eca600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec406e5e0 con 0x7f0ec406df00 2024-08-17T02:45:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.216+0000 7f0ec3e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ec4155df0 0x7f0ec40ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.216+0000 7f0ec3400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0ec4155a20 0x7f0ec406d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59730/0 (socket says 172.21.15.16:59730) 2024-08-17T02:45:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.216+0000 7f0ec3400640 1 -- 172.21.15.16:0/2445713144 learned_addr learned my addr 172.21.15.16:0/2445713144 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.216+0000 7f0ec3e00640 1 --2- 172.21.15.16:0/2445713144 >> v2:172.21.15.16:3300/0 conn(0x7f0ec4155df0 0x7f0ec40ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33034/0 (socket says 172.21.15.16:33034) 2024-08-17T02:45:20.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.216+0000 7f0ec2a00640 1 --2- 172.21.15.16:0/2445713144 >> v2:172.21.15.142:3300/0 conn(0x7f0ec406df00 0x7f0ec40adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.217+0000 7f0ec3e00640 1 -- 172.21.15.16:0/2445713144 >> v2:172.21.15.142:3300/0 conn(0x7f0ec406df00 msgr2=0x7f0ec40adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.217+0000 7f0ec3e00640 1 --2- 172.21.15.16:0/2445713144 >> v2:172.21.15.142:3300/0 conn(0x7f0ec406df00 0x7f0ec40adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.217+0000 7f0ec3e00640 1 -- 172.21.15.16:0/2445713144 >> v1:172.21.15.16:6789/0 conn(0x7f0ec4155a20 legacy=0x7f0ec406d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.217+0000 7f0ec3e00640 1 -- 172.21.15.16:0/2445713144 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec406e8c0 con 0x7f0ec4155df0 2024-08-17T02:45:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.217+0000 7f0ec2000640 1 -- 172.21.15.16:0/2445713144 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111289853 0 0) 0x7f0ec4156670 con 0x7f0ec4155a20 2024-08-17T02:45:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.218+0000 7f0ec3e00640 1 --2- 172.21.15.16:0/2445713144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4155df0 0x7f0ec40ad9b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0ec406aa40 tx=0x7f0ebc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502e00beb7af0b68 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.218+0000 7f0ec2000640 1 -- 172.21.15.16:0/2445713144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc007550 con 0x7f0ec4155df0 2024-08-17T02:45:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.218+0000 7f0ec2000640 1 -- 172.21.15.16:0/2445713144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc02d050 con 0x7f0ec4155df0 2024-08-17T02:45:20.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.218+0000 7f0ec2000640 1 -- 172.21.15.16:0/2445713144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0078e0 con 0x7f0ec4155df0 2024-08-17T02:45:20.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.218+0000 7f0eca600640 1 -- 172.21.15.16:0/2445713144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4155df0 msgr2=0x7f0ec40ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.218+0000 7f0eca600640 1 --2- 172.21.15.16:0/2445713144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4155df0 0x7f0ec40ad9b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0ec406aa40 tx=0x7f0ebc009780 comp rx=0 tx=0).stop 2024-08-17T02:45:20.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.219+0000 7f0eca600640 1 -- 172.21.15.16:0/2445713144 shutdown_connections 2024-08-17T02:45:20.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.219+0000 7f0eca600640 1 --2- 172.21.15.16:0/2445713144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4155df0 0x7f0ec40ad9b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.219+0000 7f0eca600640 1 --2- 172.21.15.16:0/2445713144 >> v2:172.21.15.142:3300/0 conn(0x7f0ec406df00 0x7f0ec40adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.219+0000 7f0eca600640 1 -- 172.21.15.16:0/2445713144 >> 172.21.15.16:0/2445713144 conn(0x7f0ec40cb9b0 msgr2=0x7f0ec40cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.219+0000 7f0eca600640 1 -- 172.21.15.16:0/2445713144 shutdown_connections 2024-08-17T02:45:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.219+0000 7f0eca600640 1 -- 172.21.15.16:0/2445713144 wait complete. 2024-08-17T02:45:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.220+0000 7f0eca600640 1 Processor -- start 2024-08-17T02:45:20.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.220+0000 7f0eca600640 1 -- start start 2024-08-17T02:45:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.221+0000 7f0eca600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ec406df00 0x7f0ec40acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.221+0000 7f0eca600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4155a20 0x7f0ec41453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.221+0000 7f0eca600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ec4155df0 0x7f0ec40aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.221+0000 7f0eca600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec40aa640 con 0x7f0ec4155a20 2024-08-17T02:45:20.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.221+0000 7f0eca600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec40a9da0 con 0x7f0ec4155df0 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.221+0000 7f0eca600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec40aa080 con 0x7f0ec406df00 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.221+0000 7f0ec2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4155a20 0x7f0ec41453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.221+0000 7f0ec2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4155a20 0x7f0ec41453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33038/0 (socket says 172.21.15.16:33038) 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.221+0000 7f0ec2a00640 1 -- 172.21.15.16:0/1412050078 learned_addr learned my addr 172.21.15.16:0/1412050078 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.221+0000 7f0ec3400640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ec406df00 0x7f0ec40acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.221+0000 7f0ec3e00640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ec4155df0 0x7f0ec40aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.222+0000 7f0ec2a00640 1 -- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ec406df00 msgr2=0x7f0ec40acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.222+0000 7f0ec2a00640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ec406df00 0x7f0ec40acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.222+0000 7f0ec2a00640 1 -- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ec4155df0 msgr2=0x7f0ec40aafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.222+0000 7f0ec2a00640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ec4155df0 0x7f0ec40aafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.222+0000 7f0ec2a00640 1 -- 172.21.15.16:0/1412050078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc009a20 con 0x7f0ec4155a20 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.222+0000 7f0ec3e00640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ec4155df0 0x7f0ec40aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:20.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.222+0000 7f0ec2a00640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4155a20 0x7f0ec41453c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009fd0 tx=0x7f0eb00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.222+0000 7f0ebaa00640 1 -- 172.21.15.16:0/1412050078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000c940 con 0x7f0ec4155a20 2024-08-17T02:45:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.222+0000 7f0ebaa00640 1 -- 172.21.15.16:0/1412050078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb000caa0 con 0x7f0ec4155a20 2024-08-17T02:45:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.222+0000 7f0ebaa00640 1 -- 172.21.15.16:0/1412050078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000ce40 con 0x7f0ec4155a20 2024-08-17T02:45:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.222+0000 7f0eca600640 1 -- 172.21.15.16:0/1412050078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec40aa360 con 0x7f0ec4155a20 2024-08-17T02:45:20.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.222+0000 7f0eca600640 1 -- 172.21.15.16:0/1412050078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec4145900 con 0x7f0ec4155a20 2024-08-17T02:45:20.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.223+0000 7f0eca600640 1 -- 172.21.15.16:0/1412050078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec40a5bd0 con 0x7f0ec4155a20 2024-08-17T02:45:20.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.224+0000 7f0ebaa00640 1 -- 172.21.15.16:0/1412050078 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f0eb0013680 con 0x7f0ec4155a20 2024-08-17T02:45:20.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.225+0000 7f0ebaa00640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0ea4065150 0x7f0ea4067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.225+0000 7f0ebaa00640 1 -- 172.21.15.16:0/1412050078 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0eb00826e0 con 0x7f0ec4155a20 2024-08-17T02:45:20.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.225+0000 7f0ec3400640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0ea4065150 0x7f0ea4067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.226+0000 7f0ec3400640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0ea4065150 0x7f0ea4067600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40011e0 tx=0x7f0eb4007ca0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:20.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.229+0000 7f0ebaa00640 1 -- 172.21.15.16:0/1412050078 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0eb00507b0 con 0x7f0ec4155a20 2024-08-17T02:45:20.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.414+0000 7f0eca600640 1 -- 172.21.15.16:0/1412050078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ec4064db0 con 0x7f0ec4155a20 2024-08-17T02:45:20.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.415+0000 7f0ebaa00640 1 -- 172.21.15.16:0/1412050078 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f0eb0053dd0 con 0x7f0ec4155a20 2024-08-17T02:45:20.417 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:20.417 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:19.450349+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.417+0000 7f0eca600640 1 -- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0ea4065150 msgr2=0x7f0ea4067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.417+0000 7f0eca600640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0ea4065150 0x7f0ea4067600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40011e0 tx=0x7f0eb4007ca0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.417+0000 7f0eca600640 1 -- 172.21.15.16:0/1412050078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4155a20 msgr2=0x7f0ec41453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.417+0000 7f0eca600640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4155a20 0x7f0ec41453c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009fd0 tx=0x7f0eb00090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.417+0000 7f0eca600640 1 -- 172.21.15.16:0/1412050078 shutdown_connections 2024-08-17T02:45:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.417+0000 7f0eca600640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0ea4065150 0x7f0ea4067600 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.417+0000 7f0eca600640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ec4155df0 0x7f0ec40aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.418+0000 7f0eca600640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ec4155a20 0x7f0ec41453c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.418+0000 7f0eca600640 1 --2- 172.21.15.16:0/1412050078 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0ec406df00 0x7f0ec40acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.418+0000 7f0eca600640 1 -- 172.21.15.16:0/1412050078 >> 172.21.15.16:0/1412050078 conn(0x7f0ec40cb9b0 msgr2=0x7f0ec4153fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:20.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.418+0000 7f0eca600640 1 -- 172.21.15.16:0/1412050078 shutdown_connections 2024-08-17T02:45:20.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.418+0000 7f0eca600640 1 -- 172.21.15.16:0/1412050078 wait complete. 2024-08-17T02:45:20.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.584+0000 7f3ca0e00640 1 Processor -- start 2024-08-17T02:45:20.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.584+0000 7f3ca0e00640 1 -- start start 2024-08-17T02:45:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.585+0000 7f3ca0e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c9c158ce0 0x7f3c9c0c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.585+0000 7f3ca0e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c9c0d08d0 0x7f3c9c1590b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.585+0000 7f3ca0e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9c0cb7c0 con 0x7f3c9c0cac30 2024-08-17T02:45:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.585+0000 7f3ca0e00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c9c0cf0c0 con 0x7f3c9c158ce0 2024-08-17T02:45:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.585+0000 7f3ca0e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c089000 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.586+0000 7f3c99e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c9c158ce0 0x7f3c9c0c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.586+0000 7f3c99e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c9c158ce0 0x7f3c9c0c68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33052/0 (socket says 172.21.15.16:33052) 2024-08-17T02:45:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.586+0000 7f3c99e00640 1 -- 172.21.15.16:0/544686392 learned_addr learned my addr 172.21.15.16:0/544686392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.586+0000 7f3c99400640 1 --2- 172.21.15.16:0/544686392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c9c0d08d0 0x7f3c9c1590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.586+0000 7f3c99e00640 1 -- 172.21.15.16:0/544686392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c9c0d08d0 msgr2=0x7f3c9c1590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.586+0000 7f3c99e00640 1 --2- 172.21.15.16:0/544686392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c9c0d08d0 0x7f3c9c1590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.586+0000 7f3c99e00640 1 -- 172.21.15.16:0/544686392 >> v1:172.21.15.142:6789/0 conn(0x7f3c9c0cac30 legacy=0x7f3c9c0cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.586+0000 7f3c99e00640 1 -- 172.21.15.16:0/544686392 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c0ceb20 con 0x7f3c9c158ce0 2024-08-17T02:45:20.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.586+0000 7f3c99400640 1 --2- 172.21.15.16:0/544686392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c9c0d08d0 0x7f3c9c1590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.587+0000 7f3c99e00640 1 --2- 172.21.15.16:0/544686392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c158ce0 0x7f3c9c0c68c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c009940 tx=0x7f3c8c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c01f5f8197aea4 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.587+0000 7f3c98a00640 1 -- 172.21.15.16:0/544686392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c02ae40 con 0x7f3c9c158ce0 2024-08-17T02:45:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.587+0000 7f3c98a00640 1 -- 172.21.15.16:0/544686392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8c032440 con 0x7f3c9c158ce0 2024-08-17T02:45:20.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.587+0000 7f3c98a00640 1 -- 172.21.15.16:0/544686392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c032820 con 0x7f3c9c158ce0 2024-08-17T02:45:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.588+0000 7f3ca0e00640 1 -- 172.21.15.16:0/544686392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c158ce0 msgr2=0x7f3c9c0c68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.588+0000 7f3ca0e00640 1 --2- 172.21.15.16:0/544686392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c158ce0 0x7f3c9c0c68c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c009940 tx=0x7f3c8c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.588+0000 7f3ca0e00640 1 -- 172.21.15.16:0/544686392 shutdown_connections 2024-08-17T02:45:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.588+0000 7f3ca0e00640 1 --2- 172.21.15.16:0/544686392 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c9c0d08d0 0x7f3c9c1590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.588+0000 7f3ca0e00640 1 --2- 172.21.15.16:0/544686392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c158ce0 0x7f3c9c0c68c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.588+0000 7f3ca0e00640 1 -- 172.21.15.16:0/544686392 >> 172.21.15.16:0/544686392 conn(0x7f3c9c0cfdc0 msgr2=0x7f3c9c0d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.588+0000 7f3ca0e00640 1 -- 172.21.15.16:0/544686392 shutdown_connections 2024-08-17T02:45:20.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.588+0000 7f3ca0e00640 1 -- 172.21.15.16:0/544686392 wait complete. 2024-08-17T02:45:20.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.589+0000 7f3ca0e00640 1 Processor -- start 2024-08-17T02:45:20.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.590+0000 7f3ca0e00640 1 -- start start 2024-08-17T02:45:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.590+0000 7f3ca0e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c9c0cac30 0x7f3c9c067be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.590+0000 7f3ca0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c0d08d0 0x7f3c9c0b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.590+0000 7f3ca0e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9c158ce0 0x7f3c9c0b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.590+0000 7f3ca0e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c0c9cf0 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.590+0000 7f3ca0e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c0c90e0 con 0x7f3c9c158ce0 2024-08-17T02:45:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.590+0000 7f3ca0e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c0c93c0 con 0x7f3c9c0cac30 2024-08-17T02:45:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c99400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c0d08d0 0x7f3c9c0b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c99400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c0d08d0 0x7f3c9c0b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33058/0 (socket says 172.21.15.16:33058) 2024-08-17T02:45:20.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c99400640 1 -- 172.21.15.16:0/2456583126 learned_addr learned my addr 172.21.15.16:0/2456583126 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c99400640 1 -- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c9c0cac30 msgr2=0x7f3c9c067be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c99e00640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c9c0cac30 0x7f3c9c067be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c9a800640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9c158ce0 0x7f3c9c0b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c99400640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c9c0cac30 0x7f3c9c067be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c99400640 1 -- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9c158ce0 msgr2=0x7f3c9c0b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c99400640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9c158ce0 0x7f3c9c0b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c99400640 1 -- 172.21.15.16:0/2456583126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c009450 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c99e00640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c9c0cac30 0x7f3c9c067be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c9a800640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9c158ce0 0x7f3c9c0b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.591+0000 7f3c99400640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c0d08d0 0x7f3c9c0b3cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800d890 tx=0x7f3c88008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.592+0000 7f3c91600640 1 -- 172.21.15.16:0/2456583126 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8800de80 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.592+0000 7f3c91600640 1 -- 172.21.15.16:0/2456583126 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8800b8d0 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.592+0000 7f3ca0e00640 1 -- 172.21.15.16:0/2456583126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9c0c96a0 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.592+0000 7f3c91600640 1 -- 172.21.15.16:0/2456583126 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8800bc90 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.592+0000 7f3ca0e00640 1 -- 172.21.15.16:0/2456583126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9c0c8d40 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.593+0000 7f3ca0e00640 1 -- 172.21.15.16:0/2456583126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c9c064db0 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.594+0000 7f3c91600640 1 -- 172.21.15.16:0/2456583126 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f3c8801e460 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.595+0000 7f3c91600640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3c7c065150 0x7f3c7c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.595+0000 7f3c91600640 1 -- 172.21.15.16:0/2456583126 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3c88082b30 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.595+0000 7f3c99e00640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3c7c065150 0x7f3c7c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.596+0000 7f3c99e00640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3c7c065150 0x7f3c7c067600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c034000 tx=0x7f3c8c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:20.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.599+0000 7f3c91600640 1 -- 172.21.15.16:0/2456583126 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f3c88054680 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.783+0000 7f3ca0e00640 1 -- 172.21.15.16:0/2456583126 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c9c169180 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.784+0000 7f3c91600640 1 -- 172.21.15.16:0/2456583126 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f3c88054210 con 0x7f3c9c0d08d0 2024-08-17T02:45:20.785 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:20.785 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:19.450349+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:20.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.786+0000 7f3ca0e00640 1 -- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3c7c065150 msgr2=0x7f3c7c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.786+0000 7f3ca0e00640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3c7c065150 0x7f3c7c067600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c034000 tx=0x7f3c8c032000 comp rx=0 tx=0).stop 2024-08-17T02:45:20.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.786+0000 7f3ca0e00640 1 -- 172.21.15.16:0/2456583126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c0d08d0 msgr2=0x7f3c9c0b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.786+0000 7f3ca0e00640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c0d08d0 0x7f3c9c0b3cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800d890 tx=0x7f3c88008040 comp rx=0 tx=0).stop 2024-08-17T02:45:20.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.786+0000 7f3ca0e00640 1 -- 172.21.15.16:0/2456583126 shutdown_connections 2024-08-17T02:45:20.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.786+0000 7f3ca0e00640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3c7c065150 0x7f3c7c067600 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.786+0000 7f3ca0e00640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c9c158ce0 0x7f3c9c0b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.786+0000 7f3ca0e00640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c9c0d08d0 0x7f3c9c0b3cd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.786+0000 7f3ca0e00640 1 --2- 172.21.15.16:0/2456583126 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3c9c0cac30 0x7f3c9c067be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.786+0000 7f3ca0e00640 1 -- 172.21.15.16:0/2456583126 >> 172.21.15.16:0/2456583126 conn(0x7f3c9c0cfdc0 msgr2=0x7f3c9c157e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:20.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.786+0000 7f3ca0e00640 1 -- 172.21.15.16:0/2456583126 shutdown_connections 2024-08-17T02:45:20.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.786+0000 7f3ca0e00640 1 -- 172.21.15.16:0/2456583126 wait complete. 2024-08-17T02:45:20.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-08-17T02:45:20.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.954+0000 7f73cda00640 1 Processor -- start 2024-08-17T02:45:20.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.954+0000 7f73cda00640 1 -- start start 2024-08-17T02:45:20.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.955+0000 7f73cda00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73c8156c50 0x7f73c80c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.956+0000 7f73cda00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c80ab130 con 0x7f73c80c8bd0 2024-08-17T02:45:20.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.956+0000 7f73cda00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c80ae5a0 con 0x7f73c80adcf0 2024-08-17T02:45:20.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.956+0000 7f73cda00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c80ae2c0 con 0x7f73c8156c50 2024-08-17T02:45:20.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.956+0000 7f73c6a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73c8156c50 0x7f73c80c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.956+0000 7f73c7400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f73c80c8bd0 0x7f73c80c8ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59736/0 (socket says 172.21.15.16:59736) 2024-08-17T02:45:20.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.956+0000 7f73c7400640 1 -- 172.21.15.16:0/1866207297 learned_addr learned my addr 172.21.15.16:0/1866207297 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:20.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.956+0000 7f73c6a00640 1 -- 172.21.15.16:0/1866207297 >> v1:172.21.15.142:6789/0 conn(0x7f73c80adcf0 legacy=0x7f73c80ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:20.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.956+0000 7f73c6a00640 1 -- 172.21.15.16:0/1866207297 >> v1:172.21.15.16:6789/0 conn(0x7f73c80c8bd0 legacy=0x7f73c80c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.956+0000 7f73c6a00640 1 -- 172.21.15.16:0/1866207297 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c80aaae0 con 0x7f73c8156c50 2024-08-17T02:45:20.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.956+0000 7f73c6000640 1 -- 172.21.15.16:0/1866207297 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955142843 0 0) 0x7f73c80ab130 con 0x7f73c80c8bd0 2024-08-17T02:45:20.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.957+0000 7f73c6a00640 1 --2- 172.21.15.16:0/1866207297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c8156c50 0x7f73c80c7cf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009850 tx=0x7f73b80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a839610c00f8cc server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:20.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.958+0000 7f73c6000640 1 -- 172.21.15.16:0/1866207297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b802ac70 con 0x7f73c8156c50 2024-08-17T02:45:20.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.958+0000 7f73c6000640 1 -- 172.21.15.16:0/1866207297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b802add0 con 0x7f73c8156c50 2024-08-17T02:45:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.958+0000 7f73c6000640 1 -- 172.21.15.16:0/1866207297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8032670 con 0x7f73c8156c50 2024-08-17T02:45:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.958+0000 7f73cda00640 1 -- 172.21.15.16:0/1866207297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c8156c50 msgr2=0x7f73c80c7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.958+0000 7f73cda00640 1 --2- 172.21.15.16:0/1866207297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c8156c50 0x7f73c80c7cf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009850 tx=0x7f73b80090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.958+0000 7f73cda00640 1 -- 172.21.15.16:0/1866207297 shutdown_connections 2024-08-17T02:45:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.958+0000 7f73cda00640 1 --2- 172.21.15.16:0/1866207297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c8156c50 0x7f73c80c7cf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.958+0000 7f73cda00640 1 -- 172.21.15.16:0/1866207297 >> 172.21.15.16:0/1866207297 conn(0x7f73c8067750 msgr2=0x7f73c80c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.958+0000 7f73cda00640 1 -- 172.21.15.16:0/1866207297 shutdown_connections 2024-08-17T02:45:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.958+0000 7f73cda00640 1 -- 172.21.15.16:0/1866207297 wait complete. 2024-08-17T02:45:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.960+0000 7f73cda00640 1 Processor -- start 2024-08-17T02:45:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.960+0000 7f73cda00640 1 -- start start 2024-08-17T02:45:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73cda00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c80adcf0 0x7f73c815e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73cda00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73c80c8bd0 0x7f73c816a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73cda00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73c8156c50 0x7f73c8157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73cda00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c8158e90 con 0x7f73c80adcf0 2024-08-17T02:45:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73cda00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c8159170 con 0x7f73c80c8bd0 2024-08-17T02:45:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73cda00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c8159450 con 0x7f73c8156c50 2024-08-17T02:45:20.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73c6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c80adcf0 0x7f73c815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73c6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c80adcf0 0x7f73c815e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33074/0 (socket says 172.21.15.16:33074) 2024-08-17T02:45:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73c6a00640 1 -- 172.21.15.16:0/3480530957 learned_addr learned my addr 172.21.15.16:0/3480530957 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73c6a00640 1 -- 172.21.15.16:0/3480530957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73c8156c50 msgr2=0x7f73c8157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73c6a00640 1 --2- 172.21.15.16:0/3480530957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73c8156c50 0x7f73c8157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73c6a00640 1 -- 172.21.15.16:0/3480530957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73c80c8bd0 msgr2=0x7f73c816a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:20.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73c6a00640 1 --2- 172.21.15.16:0/3480530957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73c80c8bd0 0x7f73c816a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:20.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.961+0000 7f73c6a00640 1 -- 172.21.15.16:0/3480530957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b8009390 con 0x7f73c80adcf0 2024-08-17T02:45:20.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.962+0000 7f73c6a00640 1 --2- 172.21.15.16:0/3480530957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c80adcf0 0x7f73c815e630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009360 tx=0x7f73b8032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:20.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.962+0000 7f73bea00640 1 -- 172.21.15.16:0/3480530957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8032e20 con 0x7f73c80adcf0 2024-08-17T02:45:20.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.962+0000 7f73bea00640 1 -- 172.21.15.16:0/3480530957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b8031a60 con 0x7f73c80adcf0 2024-08-17T02:45:20.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.962+0000 7f73bea00640 1 -- 172.21.15.16:0/3480530957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8031e00 con 0x7f73c80adcf0 2024-08-17T02:45:20.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.962+0000 7f73cda00640 1 -- 172.21.15.16:0/3480530957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c8159730 con 0x7f73c80adcf0 2024-08-17T02:45:20.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.962+0000 7f73cda00640 1 -- 172.21.15.16:0/3480530957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c816a550 con 0x7f73c80adcf0 2024-08-17T02:45:20.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.963+0000 7f73cda00640 1 -- 172.21.15.16:0/3480530957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c8071050 con 0x7f73c80adcf0 2024-08-17T02:45:20.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.964+0000 7f73bea00640 1 -- 172.21.15.16:0/3480530957 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f73b8043030 con 0x7f73c80adcf0 2024-08-17T02:45:20.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.965+0000 7f73bea00640 1 --2- 172.21.15.16:0/3480530957 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f73a4065180 0x7f73a4067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:20.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.965+0000 7f73bea00640 1 -- 172.21.15.16:0/3480530957 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f73b8041070 con 0x7f73c80adcf0 2024-08-17T02:45:20.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.965+0000 7f73be000640 1 --2- 172.21.15.16:0/3480530957 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f73a4065180 0x7f73a4067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:20.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.966+0000 7f73be000640 1 --2- 172.21.15.16:0/3480530957 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f73a4065180 0x7f73a4067630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f73b0007fd0 tx=0x7f73b0007a90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:20.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:20.969+0000 7f73bea00640 1 -- 172.21.15.16:0/3480530957 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f73b8070ac0 con 0x7f73c80adcf0 2024-08-17T02:45:21.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:21.196+0000 7f73cda00640 1 -- 172.21.15.16:0/3480530957 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f73c806ced0 con 0x7f73c80adcf0 2024-08-17T02:45:22.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.039+0000 7f73bea00640 1 -- 172.21.15.16:0/3480530957 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f73b804b0b0 con 0x7f73c80adcf0 2024-08-17T02:45:22.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.042+0000 7f73cda00640 1 -- 172.21.15.16:0/3480530957 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f73a4065180 msgr2=0x7f73a4067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:22.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.042+0000 7f73cda00640 1 --2- 172.21.15.16:0/3480530957 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f73a4065180 0x7f73a4067630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f73b0007fd0 tx=0x7f73b0007a90 comp rx=0 tx=0).stop 2024-08-17T02:45:22.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.042+0000 7f73cda00640 1 -- 172.21.15.16:0/3480530957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c80adcf0 msgr2=0x7f73c815e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:22.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.042+0000 7f73cda00640 1 --2- 172.21.15.16:0/3480530957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c80adcf0 0x7f73c815e630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f73b8009360 tx=0x7f73b8032000 comp rx=0 tx=0).stop 2024-08-17T02:45:22.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.042+0000 7f73cda00640 1 -- 172.21.15.16:0/3480530957 shutdown_connections 2024-08-17T02:45:22.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.042+0000 7f73cda00640 1 --2- 172.21.15.16:0/3480530957 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f73a4065180 0x7f73a4067630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:22.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.042+0000 7f73cda00640 1 --2- 172.21.15.16:0/3480530957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f73c8156c50 0x7f73c8157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:22.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.042+0000 7f73cda00640 1 --2- 172.21.15.16:0/3480530957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73c80c8bd0 0x7f73c816a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:22.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.042+0000 7f73cda00640 1 --2- 172.21.15.16:0/3480530957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73c80adcf0 0x7f73c815e630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:22.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.042+0000 7f73cda00640 1 -- 172.21.15.16:0/3480530957 >> 172.21.15.16:0/3480530957 conn(0x7f73c8067750 msgr2=0x7f73c806c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:22.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.042+0000 7f73cda00640 1 -- 172.21.15.16:0/3480530957 shutdown_connections 2024-08-17T02:45:22.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.042+0000 7f73cda00640 1 -- 172.21.15.16:0/3480530957 wait complete. 2024-08-17T02:45:22.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-08-17T02:45:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.218+0000 7f95ed600640 1 Processor -- start 2024-08-17T02:45:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.218+0000 7f95ed600640 1 -- start start 2024-08-17T02:45:22.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.219+0000 7f95ed600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95e80c92e0 0x7f95e8160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.219+0000 7f95ed600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e81598f0 con 0x7f95e80c7db0 2024-08-17T02:45:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.219+0000 7f95ed600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e8159bd0 con 0x7f95e8155a80 2024-08-17T02:45:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.219+0000 7f95ed600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e8159eb0 con 0x7f95e80c92e0 2024-08-17T02:45:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.219+0000 7f95e7400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95e80c92e0 0x7f95e8160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.219+0000 7f95e6000640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f95e80c7db0 0x7f95e80c8bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59740/0 (socket says 172.21.15.16:59740) 2024-08-17T02:45:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.219+0000 7f95e6000640 1 -- 172.21.15.16:0/286956982 learned_addr learned my addr 172.21.15.16:0/286956982 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.220+0000 7f95e7400640 1 -- 172.21.15.16:0/286956982 >> v1:172.21.15.142:6789/0 conn(0x7f95e8155a80 legacy=0x7f95e806dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.220+0000 7f95e7400640 1 -- 172.21.15.16:0/286956982 >> v1:172.21.15.16:6789/0 conn(0x7f95e80c7db0 legacy=0x7f95e80c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.220+0000 7f95e7400640 1 -- 172.21.15.16:0/286956982 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e815a190 con 0x7f95e80c92e0 2024-08-17T02:45:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.220+0000 7f95e5600640 1 -- 172.21.15.16:0/286956982 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302281133 0 0) 0x7f95e81598f0 con 0x7f95e80c7db0 2024-08-17T02:45:22.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.221+0000 7f95e7400640 1 --2- 172.21.15.16:0/286956982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e80c92e0 0x7f95e8160b90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009e60 tx=0x7f95e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31e907c6ac94501 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:22.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.221+0000 7f95e5600640 1 -- 172.21.15.16:0/286956982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0030550 con 0x7f95e80c92e0 2024-08-17T02:45:22.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.221+0000 7f95e5600640 1 -- 172.21.15.16:0/286956982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e00306b0 con 0x7f95e80c92e0 2024-08-17T02:45:22.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.221+0000 7f95e5600640 1 -- 172.21.15.16:0/286956982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0030a50 con 0x7f95e80c92e0 2024-08-17T02:45:22.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.222+0000 7f95ed600640 1 -- 172.21.15.16:0/286956982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e80c92e0 msgr2=0x7f95e8160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:22.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.222+0000 7f95ed600640 1 --2- 172.21.15.16:0/286956982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e80c92e0 0x7f95e8160b90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009e60 tx=0x7f95e0009780 comp rx=0 tx=0).stop 2024-08-17T02:45:22.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.222+0000 7f95ed600640 1 -- 172.21.15.16:0/286956982 shutdown_connections 2024-08-17T02:45:22.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.222+0000 7f95ed600640 1 --2- 172.21.15.16:0/286956982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e80c92e0 0x7f95e8160b90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:22.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.222+0000 7f95ed600640 1 -- 172.21.15.16:0/286956982 >> 172.21.15.16:0/286956982 conn(0x7f95e8067750 msgr2=0x7f95e80c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.222+0000 7f95ed600640 1 -- 172.21.15.16:0/286956982 shutdown_connections 2024-08-17T02:45:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.222+0000 7f95ed600640 1 -- 172.21.15.16:0/286956982 wait complete. 2024-08-17T02:45:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.223+0000 7f95ed600640 1 Processor -- start 2024-08-17T02:45:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.223+0000 7f95ed600640 1 -- start start 2024-08-17T02:45:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.224+0000 7f95ed600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95e80c7db0 0x7f95e814a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.224+0000 7f95ed600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e80c92e0 0x7f95e814bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.224+0000 7f95ed600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e8155a80 0x7f95e814dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.224+0000 7f95ed600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e814a9b0 con 0x7f95e8155a80 2024-08-17T02:45:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.224+0000 7f95ed600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e814c3e0 con 0x7f95e80c92e0 2024-08-17T02:45:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.224+0000 7f95ed600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e814d220 con 0x7f95e80c7db0 2024-08-17T02:45:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.224+0000 7f95e7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e8155a80 0x7f95e814dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.224+0000 7f95e7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e8155a80 0x7f95e814dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33090/0 (socket says 172.21.15.16:33090) 2024-08-17T02:45:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.224+0000 7f95e7400640 1 -- 172.21.15.16:0/85419282 learned_addr learned my addr 172.21.15.16:0/85419282 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95e6000640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e80c92e0 0x7f95e814bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95e6a00640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95e80c7db0 0x7f95e814a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95e7400640 1 -- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95e80c7db0 msgr2=0x7f95e814a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95e7400640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95e80c7db0 0x7f95e814a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95e7400640 1 -- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e80c92e0 msgr2=0x7f95e814bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95e7400640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e80c92e0 0x7f95e814bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95e7400640 1 -- 172.21.15.16:0/85419282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e0009a20 con 0x7f95e8155a80 2024-08-17T02:45:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95e6a00640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95e80c7db0 0x7f95e814a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:22.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95e6000640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e80c92e0 0x7f95e814bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95e7400640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e8155a80 0x7f95e814dbf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f95e00099f0 tx=0x7f95e0038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95de000640 1 -- 172.21.15.16:0/85419282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0032c70 con 0x7f95e8155a80 2024-08-17T02:45:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95de000640 1 -- 172.21.15.16:0/85419282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e0041070 con 0x7f95e8155a80 2024-08-17T02:45:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95de000640 1 -- 172.21.15.16:0/85419282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0046430 con 0x7f95e8155a80 2024-08-17T02:45:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.225+0000 7f95ed600640 1 -- 172.21.15.16:0/85419282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e814e130 con 0x7f95e8155a80 2024-08-17T02:45:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.226+0000 7f95ed600640 1 -- 172.21.15.16:0/85419282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e814c970 con 0x7f95e8155a80 2024-08-17T02:45:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.227+0000 7f95ed600640 1 -- 172.21.15.16:0/85419282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e8071050 con 0x7f95e8155a80 2024-08-17T02:45:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.227+0000 7f95de000640 1 -- 172.21.15.16:0/85419282 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f95e0043030 con 0x7f95e8155a80 2024-08-17T02:45:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.228+0000 7f95de000640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f95c8065150 0x7f95c8067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.228+0000 7f95de000640 1 -- 172.21.15.16:0/85419282 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f95e00a4090 con 0x7f95e8155a80 2024-08-17T02:45:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.228+0000 7f95e6a00640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f95c8065150 0x7f95c8067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:22.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.229+0000 7f95e6a00640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f95c8065150 0x7f95c8067600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f95e8152940 tx=0x7f95d8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:22.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.232+0000 7f95de000640 1 -- 172.21.15.16:0/85419282 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f95e00720b0 con 0x7f95e8155a80 2024-08-17T02:45:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:22.456+0000 7f95ed600640 1 -- 172.21.15.16:0/85419282 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f95e806b010 con 0x7f95e8155a80 2024-08-17T02:45:23.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.046+0000 7f95de000640 1 -- 172.21.15.16:0/85419282 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f95e00756d0 con 0x7f95e8155a80 2024-08-17T02:45:23.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.049+0000 7f95ed600640 1 -- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f95c8065150 msgr2=0x7f95c8067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.049+0000 7f95ed600640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f95c8065150 0x7f95c8067600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f95e8152940 tx=0x7f95d8009040 comp rx=0 tx=0).stop 2024-08-17T02:45:23.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.049+0000 7f95ed600640 1 -- 172.21.15.16:0/85419282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e8155a80 msgr2=0x7f95e814dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.049+0000 7f95ed600640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e8155a80 0x7f95e814dbf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f95e00099f0 tx=0x7f95e0038000 comp rx=0 tx=0).stop 2024-08-17T02:45:23.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.049+0000 7f95ed600640 1 -- 172.21.15.16:0/85419282 shutdown_connections 2024-08-17T02:45:23.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.049+0000 7f95ed600640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f95c8065150 0x7f95c8067600 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.049+0000 7f95ed600640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95e8155a80 0x7f95e814dbf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.049+0000 7f95ed600640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95e80c92e0 0x7f95e814bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.049+0000 7f95ed600640 1 --2- 172.21.15.16:0/85419282 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f95e80c7db0 0x7f95e814a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.049+0000 7f95ed600640 1 -- 172.21.15.16:0/85419282 >> 172.21.15.16:0/85419282 conn(0x7f95e8067750 msgr2=0x7f95e80afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:23.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.049+0000 7f95ed600640 1 -- 172.21.15.16:0/85419282 shutdown_connections 2024-08-17T02:45:23.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.050+0000 7f95ed600640 1 -- 172.21.15.16:0/85419282 wait complete. 2024-08-17T02:45:23.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:45:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.224+0000 7f1d7ec00640 1 Processor -- start 2024-08-17T02:45:23.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.224+0000 7f1d7ec00640 1 -- start start 2024-08-17T02:45:23.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.226+0000 7f1d7ec00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d781401b0 0x7f1d780caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.226+0000 7f1d7ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d780b1440 0x7f1d780c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.226+0000 7f1d7ec00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d780caa80 con 0x7f1d78140580 2024-08-17T02:45:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.226+0000 7f1d7ec00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d780cb450 con 0x7f1d781401b0 2024-08-17T02:45:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.226+0000 7f1d7ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d780c9e70 con 0x7f1d780b1440 2024-08-17T02:45:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.226+0000 7f1d77e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d781401b0 0x7f1d780caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.226+0000 7f1d6fe00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1d78140580 0x7f1d780b0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59754/0 (socket says 172.21.15.16:59754) 2024-08-17T02:45:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.226+0000 7f1d6fe00640 1 -- 172.21.15.16:0/2780228298 learned_addr learned my addr 172.21.15.16:0/2780228298 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.226+0000 7f1d7cc00640 1 --2- 172.21.15.16:0/2780228298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d780b1440 0x7f1d780c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.226+0000 7f1d77e00640 1 -- 172.21.15.16:0/2780228298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d780b1440 msgr2=0x7f1d780c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.227+0000 7f1d77e00640 1 --2- 172.21.15.16:0/2780228298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d780b1440 0x7f1d780c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.227+0000 7f1d77e00640 1 -- 172.21.15.16:0/2780228298 >> v1:172.21.15.16:6789/0 conn(0x7f1d78140580 legacy=0x7f1d780b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.227+0000 7f1d77e00640 1 -- 172.21.15.16:0/2780228298 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d780ca150 con 0x7f1d781401b0 2024-08-17T02:45:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.227+0000 7f1d77400640 1 -- 172.21.15.16:0/2780228298 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450869773 0 0) 0x7f1d780caa80 con 0x7f1d78140580 2024-08-17T02:45:23.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.228+0000 7f1d77e00640 1 --2- 172.21.15.16:0/2780228298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d781401b0 0x7f1d780caf10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009850 tx=0x7f1d680090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9dd20668195e91 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.228+0000 7f1d77400640 1 -- 172.21.15.16:0/2780228298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6802ad00 con 0x7f1d781401b0 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.228+0000 7f1d77400640 1 -- 172.21.15.16:0/2780228298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6802ae60 con 0x7f1d781401b0 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.228+0000 7f1d77400640 1 -- 172.21.15.16:0/2780228298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68032670 con 0x7f1d781401b0 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.228+0000 7f1d7ec00640 1 -- 172.21.15.16:0/2780228298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d781401b0 msgr2=0x7f1d780caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.228+0000 7f1d7ec00640 1 --2- 172.21.15.16:0/2780228298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d781401b0 0x7f1d780caf10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009850 tx=0x7f1d680090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.228+0000 7f1d7ec00640 1 -- 172.21.15.16:0/2780228298 shutdown_connections 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.228+0000 7f1d7ec00640 1 --2- 172.21.15.16:0/2780228298 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d780b1440 0x7f1d780c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.228+0000 7f1d7ec00640 1 --2- 172.21.15.16:0/2780228298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d781401b0 0x7f1d780caf10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.228+0000 7f1d7ec00640 1 -- 172.21.15.16:0/2780228298 >> 172.21.15.16:0/2780228298 conn(0x7f1d780cb9b0 msgr2=0x7f1d780cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.229+0000 7f1d7ec00640 1 -- 172.21.15.16:0/2780228298 shutdown_connections 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.229+0000 7f1d7ec00640 1 -- 172.21.15.16:0/2780228298 wait complete. 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.230+0000 7f1d7ec00640 1 Processor -- start 2024-08-17T02:45:23.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.230+0000 7f1d7ec00640 1 -- start start 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.231+0000 7f1d7ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d780b1440 0x7f1d781566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.231+0000 7f1d7ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d781401b0 0x7f1d780a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.231+0000 7f1d7ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d78140580 0x7f1d780a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.231+0000 7f1d7ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d78166e30 con 0x7f1d780b1440 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.231+0000 7f1d7ec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d78167110 con 0x7f1d78140580 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.231+0000 7f1d7ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d781673f0 con 0x7f1d781401b0 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.231+0000 7f1d77e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d780b1440 0x7f1d781566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.231+0000 7f1d77e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d780b1440 0x7f1d781566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33108/0 (socket says 172.21.15.16:33108) 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.231+0000 7f1d77e00640 1 -- 172.21.15.16:0/1082318312 learned_addr learned my addr 172.21.15.16:0/1082318312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d6fe00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d781401b0 0x7f1d780a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d77e00640 1 -- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d781401b0 msgr2=0x7f1d780a64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d7cc00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d78140580 0x7f1d780a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d77e00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d781401b0 0x7f1d780a64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d77e00640 1 -- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d78140580 msgr2=0x7f1d780a8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d77e00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d78140580 0x7f1d780a8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d77e00640 1 -- 172.21.15.16:0/1082318312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68009390 con 0x7f1d780b1440 2024-08-17T02:45:23.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d6fe00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d781401b0 0x7f1d780a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:23.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d7cc00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d78140580 0x7f1d780a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:23.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d77e00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d780b1440 0x7f1d781566b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009850 tx=0x7f1d68032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:23.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d6f400640 1 -- 172.21.15.16:0/1082318312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68031780 con 0x7f1d780b1440 2024-08-17T02:45:23.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d6f400640 1 -- 172.21.15.16:0/1082318312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d68031900 con 0x7f1d780b1440 2024-08-17T02:45:23.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d7ec00640 1 -- 172.21.15.16:0/1082318312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d781676d0 con 0x7f1d780b1440 2024-08-17T02:45:23.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d6f400640 1 -- 172.21.15.16:0/1082318312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68031d00 con 0x7f1d780b1440 2024-08-17T02:45:23.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.232+0000 7f1d7ec00640 1 -- 172.21.15.16:0/1082318312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d780a6a30 con 0x7f1d780b1440 2024-08-17T02:45:23.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.233+0000 7f1d7ec00640 1 -- 172.21.15.16:0/1082318312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d78064db0 con 0x7f1d780b1440 2024-08-17T02:45:23.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.234+0000 7f1d6f400640 1 -- 172.21.15.16:0/1082318312 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1d68043030 con 0x7f1d780b1440 2024-08-17T02:45:23.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.235+0000 7f1d6f400640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1d54065150 0x7f1d54067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.235+0000 7f1d6f400640 1 -- 172.21.15.16:0/1082318312 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1d68041070 con 0x7f1d780b1440 2024-08-17T02:45:23.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.235+0000 7f1d6fe00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1d54065150 0x7f1d54067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.236+0000 7f1d6fe00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1d54065150 0x7f1d54067600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d780a6d40 tx=0x7f1d60009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:23.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.239+0000 7f1d6f400640 1 -- 172.21.15.16:0/1082318312 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1d68070b90 con 0x7f1d780b1440 2024-08-17T02:45:23.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.458+0000 7f1d7ec00640 1 -- 172.21.15.16:0/1082318312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d780ab0a0 con 0x7f1d780b1440 2024-08-17T02:45:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.459+0000 7f1d6f400640 1 -- 172.21.15.16:0/1082318312 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f1d6804b090 con 0x7f1d780b1440 2024-08-17T02:45:23.460 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:23.460 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/111049205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":111049205},{"type":"v1","addr":"172.21.15.142:6836","nonce":111049205}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2692359383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2692359383},{"type":"v1","addr":"172.21.15.142:6837","nonce":2692359383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3179355872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3179355872},{"type":"v1","addr":"172.21.15.16:6833","nonce":3179355872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:45:19.450658+0000","modified":"2024-08-17T02:45:23.039406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2140506421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2140506421},{"type":"v1","addr":"172.21.15.16:6835","nonce":2140506421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:45:23.460 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-08-17T02:45:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.461+0000 7f1d7ec00640 1 -- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1d54065150 msgr2=0x7f1d54067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.461+0000 7f1d7ec00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1d54065150 0x7f1d54067600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d780a6d40 tx=0x7f1d60009040 comp rx=0 tx=0).stop 2024-08-17T02:45:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.461+0000 7f1d7ec00640 1 -- 172.21.15.16:0/1082318312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d780b1440 msgr2=0x7f1d781566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.461+0000 7f1d7ec00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d780b1440 0x7f1d781566b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009850 tx=0x7f1d68032000 comp rx=0 tx=0).stop 2024-08-17T02:45:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.461+0000 7f1d7ec00640 1 -- 172.21.15.16:0/1082318312 shutdown_connections 2024-08-17T02:45:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.461+0000 7f1d7ec00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1d54065150 0x7f1d54067600 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.461+0000 7f1d7ec00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d78140580 0x7f1d780a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.461+0000 7f1d7ec00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d781401b0 0x7f1d780a64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.461+0000 7f1d7ec00640 1 --2- 172.21.15.16:0/1082318312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d780b1440 0x7f1d781566b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.461+0000 7f1d7ec00640 1 -- 172.21.15.16:0/1082318312 >> 172.21.15.16:0/1082318312 conn(0x7f1d780cb9b0 msgr2=0x7f1d780c7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:23.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.461+0000 7f1d7ec00640 1 -- 172.21.15.16:0/1082318312 shutdown_connections 2024-08-17T02:45:23.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.461+0000 7f1d7ec00640 1 -- 172.21.15.16:0/1082318312 wait complete. 2024-08-17T02:45:23.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.627+0000 7fc19fe00640 1 Processor -- start 2024-08-17T02:45:23.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.628+0000 7fc19fe00640 1 -- start start 2024-08-17T02:45:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.629+0000 7fc19fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc1a0156ad0 0x7fc1a00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.629+0000 7fc19fe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1a006dc80 0x7fc1a01577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.629+0000 7fc19fe00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a00caaf0 con 0x7fc1a0156ea0 2024-08-17T02:45:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.629+0000 7fc19fe00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a00ca900 con 0x7fc1a006dc80 2024-08-17T02:45:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.629+0000 7fc19fe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a00cc560 con 0x7fc1a0156ad0 2024-08-17T02:45:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.629+0000 7fc19f400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1a006dc80 0x7fc1a01577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.629+0000 7fc197e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc1a0156ea0 0x7fc1a006d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59768/0 (socket says 172.21.15.16:59768) 2024-08-17T02:45:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.629+0000 7fc197e00640 1 -- 172.21.15.16:0/1819194457 learned_addr learned my addr 172.21.15.16:0/1819194457 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:23.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.629+0000 7fc19ea00640 1 --2- 172.21.15.16:0/1819194457 >> v2:172.21.15.142:3300/0 conn(0x7fc1a0156ad0 0x7fc1a00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.630+0000 7fc19f400640 1 -- 172.21.15.16:0/1819194457 >> v2:172.21.15.142:3300/0 conn(0x7fc1a0156ad0 msgr2=0x7fc1a00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.630+0000 7fc19f400640 1 --2- 172.21.15.16:0/1819194457 >> v2:172.21.15.142:3300/0 conn(0x7fc1a0156ad0 0x7fc1a00cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.630+0000 7fc19f400640 1 -- 172.21.15.16:0/1819194457 >> v1:172.21.15.16:6789/0 conn(0x7fc1a0156ea0 legacy=0x7fc1a006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.630+0000 7fc19f400640 1 -- 172.21.15.16:0/1819194457 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a0157300 con 0x7fc1a006dc80 2024-08-17T02:45:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.630+0000 7fc19e000640 1 -- 172.21.15.16:0/1819194457 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658087384 0 0) 0x7fc1a00caaf0 con 0x7fc1a0156ea0 2024-08-17T02:45:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.631+0000 7fc19f400640 1 --2- 172.21.15.16:0/1819194457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a006dc80 0x7fc1a01577f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0065fd0 tx=0x7fc198009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f793b2b6a5291dfd server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.631+0000 7fc19e000640 1 -- 172.21.15.16:0/1819194457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198007550 con 0x7fc1a006dc80 2024-08-17T02:45:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.631+0000 7fc19e000640 1 -- 172.21.15.16:0/1819194457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19802d050 con 0x7fc1a006dc80 2024-08-17T02:45:23.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.631+0000 7fc19e000640 1 -- 172.21.15.16:0/1819194457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1980078e0 con 0x7fc1a006dc80 2024-08-17T02:45:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.631+0000 7fc19fe00640 1 -- 172.21.15.16:0/1819194457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a006dc80 msgr2=0x7fc1a01577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.631+0000 7fc19fe00640 1 --2- 172.21.15.16:0/1819194457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a006dc80 0x7fc1a01577f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0065fd0 tx=0x7fc198009780 comp rx=0 tx=0).stop 2024-08-17T02:45:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.631+0000 7fc19fe00640 1 -- 172.21.15.16:0/1819194457 shutdown_connections 2024-08-17T02:45:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.631+0000 7fc19fe00640 1 --2- 172.21.15.16:0/1819194457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a006dc80 0x7fc1a01577f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.631+0000 7fc19fe00640 1 --2- 172.21.15.16:0/1819194457 >> v2:172.21.15.142:3300/0 conn(0x7fc1a0156ad0 0x7fc1a00cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.631+0000 7fc19fe00640 1 -- 172.21.15.16:0/1819194457 >> 172.21.15.16:0/1819194457 conn(0x7fc1a0067750 msgr2=0x7fc1a00cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.632+0000 7fc19fe00640 1 -- 172.21.15.16:0/1819194457 shutdown_connections 2024-08-17T02:45:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.632+0000 7fc19fe00640 1 -- 172.21.15.16:0/1819194457 wait complete. 2024-08-17T02:45:23.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.633+0000 7fc19fe00640 1 Processor -- start 2024-08-17T02:45:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.633+0000 7fc19fe00640 1 -- start start 2024-08-17T02:45:23.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1a006dc80 0x7fc1a00c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0156ad0 0x7fc1a0168bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc19fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1a0156ea0 0x7fc1a01471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc19fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a0169100 con 0x7fc1a0156ad0 2024-08-17T02:45:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc19fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a0169360 con 0x7fc1a006dc80 2024-08-17T02:45:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc19fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a0169640 con 0x7fc1a0156ea0 2024-08-17T02:45:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc197e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0156ad0 0x7fc1a0168bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc197e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0156ad0 0x7fc1a0168bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33130/0 (socket says 172.21.15.16:33130) 2024-08-17T02:45:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc197e00640 1 -- 172.21.15.16:0/656470464 learned_addr learned my addr 172.21.15.16:0/656470464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc19ea00640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1a006dc80 0x7fc1a00c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc19f400640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1a0156ea0 0x7fc1a01471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc197e00640 1 -- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1a0156ea0 msgr2=0x7fc1a01471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc197e00640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1a0156ea0 0x7fc1a01471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc197e00640 1 -- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1a006dc80 msgr2=0x7fc1a00c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc197e00640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1a006dc80 0x7fc1a00c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc197e00640 1 -- 172.21.15.16:0/656470464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc198009a20 con 0x7fc1a0156ad0 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.634+0000 7fc19f400640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1a0156ea0 0x7fc1a01471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.635+0000 7fc197e00640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0156ad0 0x7fc1a0168bc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c510 tx=0x7fc18800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.635+0000 7fc196000640 1 -- 172.21.15.16:0/656470464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188005aa0 con 0x7fc1a0156ad0 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.635+0000 7fc196000640 1 -- 172.21.15.16:0/656470464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc188005c00 con 0x7fc1a0156ad0 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.635+0000 7fc196000640 1 -- 172.21.15.16:0/656470464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1880070d0 con 0x7fc1a0156ad0 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.635+0000 7fc19fe00640 1 -- 172.21.15.16:0/656470464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a0169920 con 0x7fc1a0156ad0 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.635+0000 7fc19fe00640 1 -- 172.21.15.16:0/656470464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a0169ee0 con 0x7fc1a0156ad0 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.636+0000 7fc19fe00640 1 -- 172.21.15.16:0/656470464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a016a100 con 0x7fc1a0156ad0 2024-08-17T02:45:23.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.637+0000 7fc196000640 1 -- 172.21.15.16:0/656470464 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fc188007500 con 0x7fc1a0156ad0 2024-08-17T02:45:23.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.638+0000 7fc196000640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc17c0651d0 0x7fc17c067680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.638+0000 7fc196000640 1 -- 172.21.15.16:0/656470464 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc188081970 con 0x7fc1a0156ad0 2024-08-17T02:45:23.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.638+0000 7fc19ea00640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc17c0651d0 0x7fc17c067680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.639+0000 7fc19ea00640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc17c0651d0 0x7fc17c067680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc190009d20 tx=0x7fc1900090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:23.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.642+0000 7fc196000640 1 -- 172.21.15.16:0/656470464 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc18804fa40 con 0x7fc1a0156ad0 2024-08-17T02:45:23.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.825+0000 7fc19fe00640 1 -- 172.21.15.16:0/656470464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1a0071050 con 0x7fc1a0156ad0 2024-08-17T02:45:23.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.826+0000 7fc196000640 1 -- 172.21.15.16:0/656470464 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7fc188017090 con 0x7fc1a0156ad0 2024-08-17T02:45:23.827 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.828+0000 7fc19fe00640 1 -- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc17c0651d0 msgr2=0x7fc17c067680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.828+0000 7fc19fe00640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc17c0651d0 0x7fc17c067680 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc190009d20 tx=0x7fc1900090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.828+0000 7fc19fe00640 1 -- 172.21.15.16:0/656470464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0156ad0 msgr2=0x7fc1a0168bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.828+0000 7fc19fe00640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0156ad0 0x7fc1a0168bc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c510 tx=0x7fc18800c910 comp rx=0 tx=0).stop 2024-08-17T02:45:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.828+0000 7fc19fe00640 1 -- 172.21.15.16:0/656470464 shutdown_connections 2024-08-17T02:45:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.828+0000 7fc19fe00640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc17c0651d0 0x7fc17c067680 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.828+0000 7fc19fe00640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1a0156ea0 0x7fc1a01471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.828+0000 7fc19fe00640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a0156ad0 0x7fc1a0168bc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.828+0000 7fc19fe00640 1 --2- 172.21.15.16:0/656470464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1a006dc80 0x7fc1a00c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.828+0000 7fc19fe00640 1 -- 172.21.15.16:0/656470464 >> 172.21.15.16:0/656470464 conn(0x7fc1a0067750 msgr2=0x7fc1a0069680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.828+0000 7fc19fe00640 1 -- 172.21.15.16:0/656470464 shutdown_connections 2024-08-17T02:45:23.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.828+0000 7fc19fe00640 1 -- 172.21.15.16:0/656470464 wait complete. 2024-08-17T02:45:23.844 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-08-17T02:45:23.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:45:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.995+0000 7f7731400640 1 Processor -- start 2024-08-17T02:45:23.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.996+0000 7f7731400640 1 -- start start 2024-08-17T02:45:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.997+0000 7f7731400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f772c0ce8a0 0x7f772c0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.997+0000 7f7731400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f772c0c5c40 0x7f772c0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:23.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.997+0000 7f7731400640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772c0cc890 con 0x7f772c0aed50 2024-08-17T02:45:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.997+0000 7f7731400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772c0cb230 con 0x7f772c0c5c40 2024-08-17T02:45:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.997+0000 7f7731400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c0cb4c0 con 0x7f772c0ce8a0 2024-08-17T02:45:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.997+0000 7f772be00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f772c0aed50 0x7f772c0ce190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46088/0 (socket says 172.21.15.16:46088) 2024-08-17T02:45:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.997+0000 7f772be00640 1 -- 172.21.15.16:0/2363141155 learned_addr learned my addr 172.21.15.16:0/2363141155 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.997+0000 7f7730a00640 1 --2- 172.21.15.16:0/2363141155 >> v2:172.21.15.142:3300/0 conn(0x7f772c0c5c40 0x7f772c0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.997+0000 7f7723e00640 1 --2- 172.21.15.16:0/2363141155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f772c0ce8a0 0x7f772c0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.997+0000 7f772b400640 1 -- 172.21.15.16:0/2363141155 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950206187 0 0) 0x7f772c0cc890 con 0x7f772c0aed50 2024-08-17T02:45:23.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.997+0000 7f772b400640 1 -- 172.21.15.16:0/2363141155 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77100034e0 con 0x7f772c0aed50 2024-08-17T02:45:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.998+0000 7f772b400640 1 -- 172.21.15.16:0/2363141155 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1171360884 0 0) 0x7f77100034e0 con 0x7f772c0aed50 2024-08-17T02:45:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.998+0000 7f7723e00640 1 -- 172.21.15.16:0/2363141155 >> v2:172.21.15.142:3300/0 conn(0x7f772c0c5c40 msgr2=0x7f772c0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:24.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.998+0000 7f7723e00640 1 --2- 172.21.15.16:0/2363141155 >> v2:172.21.15.142:3300/0 conn(0x7f772c0c5c40 0x7f772c0cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.998+0000 7f7723e00640 1 -- 172.21.15.16:0/2363141155 >> v1:172.21.15.16:6789/0 conn(0x7f772c0aed50 legacy=0x7f772c0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.998+0000 7f7723e00640 1 -- 172.21.15.16:0/2363141155 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c0cb7a0 con 0x7f772c0ce8a0 2024-08-17T02:45:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.998+0000 7f7730a00640 1 --2- 172.21.15.16:0/2363141155 >> v2:172.21.15.142:3300/0 conn(0x7f772c0c5c40 0x7f772c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.999+0000 7f7723e00640 1 --2- 172.21.15.16:0/2363141155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f772c0ce8a0 0x7f772c0ca7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7714009850 tx=0x7f77140090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7de331f7a67431c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.999+0000 7f772b400640 1 -- 172.21.15.16:0/2363141155 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c001470 con 0x7f772c0ce8a0 2024-08-17T02:45:24.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.999+0000 7f772b400640 1 -- 172.21.15.16:0/2363141155 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771402aa50 con 0x7f772c0ce8a0 2024-08-17T02:45:24.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:23.999+0000 7f772b400640 1 -- 172.21.15.16:0/2363141155 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771402add0 con 0x7f772c0ce8a0 2024-08-17T02:45:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.000+0000 7f7731400640 1 -- 172.21.15.16:0/2363141155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f772c0ce8a0 msgr2=0x7f772c0ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.000+0000 7f7731400640 1 --2- 172.21.15.16:0/2363141155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f772c0ce8a0 0x7f772c0ca7b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7714009850 tx=0x7f77140090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.000+0000 7f7731400640 1 -- 172.21.15.16:0/2363141155 shutdown_connections 2024-08-17T02:45:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.000+0000 7f7731400640 1 --2- 172.21.15.16:0/2363141155 >> v2:172.21.15.142:3300/0 conn(0x7f772c0c5c40 0x7f772c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.000+0000 7f7731400640 1 --2- 172.21.15.16:0/2363141155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f772c0ce8a0 0x7f772c0ca7b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.000+0000 7f7731400640 1 -- 172.21.15.16:0/2363141155 >> 172.21.15.16:0/2363141155 conn(0x7f772c098e30 msgr2=0x7f772c06eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.000+0000 7f7731400640 1 -- 172.21.15.16:0/2363141155 shutdown_connections 2024-08-17T02:45:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.000+0000 7f7731400640 1 -- 172.21.15.16:0/2363141155 wait complete. 2024-08-17T02:45:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.001+0000 7f7731400640 1 Processor -- start 2024-08-17T02:45:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.002+0000 7f7731400640 1 -- start start 2024-08-17T02:45:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.002+0000 7f7731400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f772c0aed50 0x7f772c14f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.002+0000 7f7731400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f772c0c5c40 0x7f772c0b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.002+0000 7f7731400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f772c0ce8a0 0x7f772c0adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.002+0000 7f7731400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c14fbf0 con 0x7f772c0ce8a0 2024-08-17T02:45:24.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.002+0000 7f7731400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c0ad840 con 0x7f772c0c5c40 2024-08-17T02:45:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.002+0000 7f7731400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c0acc30 con 0x7f772c0aed50 2024-08-17T02:45:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.003+0000 7f7730a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f772c0ce8a0 0x7f772c0adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.003+0000 7f7730a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f772c0ce8a0 0x7f772c0adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57832/0 (socket says 172.21.15.16:57832) 2024-08-17T02:45:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.003+0000 7f7730a00640 1 -- 172.21.15.16:0/656295613 learned_addr learned my addr 172.21.15.16:0/656295613 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.003+0000 7f7723e00640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f772c0c5c40 0x7f772c0b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.003+0000 7f772be00640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f772c0aed50 0x7f772c14f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.003+0000 7f7730a00640 1 -- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f772c0aed50 msgr2=0x7f772c14f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.003+0000 7f7730a00640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f772c0aed50 0x7f772c14f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.003+0000 7f7730a00640 1 -- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f772c0c5c40 msgr2=0x7f772c0b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.003+0000 7f7730a00640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f772c0c5c40 0x7f772c0b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.003+0000 7f7730a00640 1 -- 172.21.15.16:0/656295613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7724009a20 con 0x7f772c0ce8a0 2024-08-17T02:45:24.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.003+0000 7f7723e00640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f772c0c5c40 0x7f772c0b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.003+0000 7f7730a00640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f772c0ce8a0 0x7f772c0adcd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7724009f20 tx=0x7f772400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.004+0000 7f7723400640 1 -- 172.21.15.16:0/656295613 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772400ce10 con 0x7f772c0ce8a0 2024-08-17T02:45:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.004+0000 7f7723400640 1 -- 172.21.15.16:0/656295613 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772400f040 con 0x7f772c0ce8a0 2024-08-17T02:45:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.004+0000 7f7723400640 1 -- 172.21.15.16:0/656295613 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77240146d0 con 0x7f772c0ce8a0 2024-08-17T02:45:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.004+0000 7f7731400640 1 -- 172.21.15.16:0/656295613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7714009390 con 0x7f772c0ce8a0 2024-08-17T02:45:24.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.004+0000 7f7731400640 1 -- 172.21.15.16:0/656295613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772c0ad1f0 con 0x7f772c0ce8a0 2024-08-17T02:45:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.005+0000 7f7731400640 1 -- 172.21.15.16:0/656295613 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772c0ad9b0 con 0x7f772c0ce8a0 2024-08-17T02:45:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.006+0000 7f7723400640 1 -- 172.21.15.16:0/656295613 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f7724003020 con 0x7f772c0ce8a0 2024-08-17T02:45:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.006+0000 7f7723400640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7708065180 0x7f7708067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.006+0000 7f7723400640 1 -- 172.21.15.16:0/656295613 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7724082f00 con 0x7f772c0ce8a0 2024-08-17T02:45:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.007+0000 7f772be00640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7708065180 0x7f7708067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.007+0000 7f772be00640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7708065180 0x7f7708067630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f771c0015a0 tx=0x7f771c0014b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:24.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.010+0000 7f7723400640 1 -- 172.21.15.16:0/656295613 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7724050f20 con 0x7f772c0ce8a0 2024-08-17T02:45:24.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.190+0000 7f7731400640 1 -- 172.21.15.16:0/656295613 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f772c071050 con 0x7f7708065180 2024-08-17T02:45:24.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.199+0000 7f7723400640 1 -- 172.21.15.16:0/656295613 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253956 (secure 0 0 0) 0x7f772c071050 con 0x7f7708065180 2024-08-17T02:45:24.203 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:24.205 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-17T02:45:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.203+0000 7f7731400640 1 -- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7708065180 msgr2=0x7f7708067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.203+0000 7f7731400640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7708065180 0x7f7708067630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f771c0015a0 tx=0x7f771c0014b0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.203+0000 7f7731400640 1 -- 172.21.15.16:0/656295613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f772c0ce8a0 msgr2=0x7f772c0adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.203+0000 7f7731400640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f772c0ce8a0 0x7f772c0adcd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7724009f20 tx=0x7f772400b040 comp rx=0 tx=0).stop 2024-08-17T02:45:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.203+0000 7f7731400640 1 -- 172.21.15.16:0/656295613 shutdown_connections 2024-08-17T02:45:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.203+0000 7f7731400640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7708065180 0x7f7708067630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.203+0000 7f7731400640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f772c0ce8a0 0x7f772c0adcd0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.203+0000 7f7731400640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f772c0c5c40 0x7f772c0b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.203+0000 7f7731400640 1 --2- 172.21.15.16:0/656295613 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f772c0aed50 0x7f772c14f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.203+0000 7f7731400640 1 -- 172.21.15.16:0/656295613 >> 172.21.15.16:0/656295613 conn(0x7f772c098e30 msgr2=0x7f772c0ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.203+0000 7f7731400640 1 -- 172.21.15.16:0/656295613 shutdown_connections 2024-08-17T02:45:24.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.203+0000 7f7731400640 1 -- 172.21.15.16:0/656295613 wait complete. 2024-08-17T02:45:24.220 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-08-17T02:45:22.235894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398808,"kb_used_data":2496,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752575912,"statfs":{"total":773094113280,"available":770637733888,"internally_reserved":0,"allocated":2555904,"data_stored":1593898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.215256"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.016435+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:44:52.129209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.025868+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:55:24.029624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.019578+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:44:52.129209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.024714+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.018890+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:44:52.129209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.021467+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.022064+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:16.013192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.025177+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:44:52.129209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.021754+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:41:31.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.025183+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:25:12.592566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.019332+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:41:31.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.017018+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:39:06.171313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.022261+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:03:27.020817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.018419+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.016225+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:16.013192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.018381+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:41:31.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-08-17T02:45:17.874560+0000","last_change":"2024-08-17T02:45:16.001117+0000","last_active":"2024-08-17T02:45:17.874560+0000","last_peered":"2024-08-17T02:45:17.874560+0000","last_clean":"2024-08-17T02:45:17.874560+0000","last_became_active":"2024-08-17T02:45:16.000788+0000","last_became_peered":"2024-08-17T02:45:16.000788+0000","last_unstale":"2024-08-17T02:45:17.874560+0000","last_undegraded":"2024-08-17T02:45:17.874560+0000","last_fullsized":"2024-08-17T02:45:17.874560+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_clean_scrub_stamp":"2024-08-17T02:45:14.986707+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:27:37.295374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.021728+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:41:31.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.022435+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:44:52.129209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.021659+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:39:06.171313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.023911+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.016217+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:39:06.171313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.022116+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:16.013192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.017066+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:16.013192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.025652+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:03:27.020817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.020264+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:16.013192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.017030+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:41:31.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.019128+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:28:16.013192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.023607+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.021032+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.022226+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:44:52.129209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.016243+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:41:31.510009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-08-17T02:45:19.012559+0000","last_change":"2024-08-17T02:45:19.012559+0000","last_active":"2024-08-17T02:45:19.012559+0000","last_peered":"2024-08-17T02:45:19.012559+0000","last_clean":"2024-08-17T02:45:19.012559+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:19.012559+0000","last_undegraded":"2024-08-17T02:45:19.012559+0000","last_fullsized":"2024-08-17T02:45:19.012559+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2024-08-17T02:45:18.010720+0000","last_change":"2024-08-17T02:45:18.017453+0000","last_active":"2024-08-17T02:45:18.010720+0000","last_peered":"2024-08-17T02:45:18.010720+0000","last_clean":"2024-08-17T02:45:18.010720+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-17T02:45:18.010720+0000","last_undegraded":"2024-08-17T02:45:18.010720+0000","last_fullsized":"2024-08-17T02:45:18.010720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:44:52.129209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":167936,"data_stored":50194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8780000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.202}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.4530000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.798}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574850,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":167936,"data_stored":50194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":761856,"data_stored":640562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]}]},{"osd":4,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297536,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074304,"statfs":{"total":96636764160,"available":96332087296,"internally_reserved":0,"allocated":180224,"data_stored":55999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301680,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070160,"statfs":{"total":96636764160,"available":96327843840,"internally_reserved":0,"allocated":167936,"data_stored":50194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]}]},{"osd":2,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301696,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070144,"statfs":{"total":96636764160,"available":96327827456,"internally_reserved":0,"allocated":180224,"data_stored":55999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.869}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8109999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574850,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302696,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069144,"statfs":{"total":96636764160,"available":96326803456,"internally_reserved":0,"allocated":761856,"data_stored":640562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301680,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070160,"statfs":{"total":96636764160,"available":96327843840,"internally_reserved":0,"allocated":167936,"data_stored":50194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:45:24.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:45:24.325 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-08-17T02:45:24.323+0000 7f5694400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-17T02:45:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.410+0000 7f9e86c00640 1 Processor -- start 2024-08-17T02:45:24.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.410+0000 7f9e86c00640 1 -- start start 2024-08-17T02:45:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.411+0000 7f9e86c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9e800aed50 0x7f9e800ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.411+0000 7f9e86c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e800ce6d0 0x7f9e800ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.411+0000 7f9e86c00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e800cc890 con 0x7f9e800cacf0 2024-08-17T02:45:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.411+0000 7f9e86c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e800ceb20 con 0x7f9e800aed50 2024-08-17T02:45:24.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.411+0000 7f9e86c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800cb4c0 con 0x7f9e800ce6d0 2024-08-17T02:45:24.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.411+0000 7f9e86200640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9e800cacf0 0x7f9e800c4c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46098/0 (socket says 172.21.15.16:46098) 2024-08-17T02:45:24.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.411+0000 7f9e86200640 1 -- 172.21.15.16:0/2053598357 learned_addr learned my addr 172.21.15.16:0/2053598357 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:24.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.411+0000 7f9e84e00640 1 --2- 172.21.15.16:0/2053598357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e800ce6d0 0x7f9e800ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.411+0000 7f9e85800640 1 --2- 172.21.15.16:0/2053598357 >> v2:172.21.15.142:3300/0 conn(0x7f9e800aed50 0x7f9e800ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.412+0000 7f9e7fe00640 1 -- 172.21.15.16:0/2053598357 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330763448 0 0) 0x7f9e800cc890 con 0x7f9e800cacf0 2024-08-17T02:45:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.412+0000 7f9e7fe00640 1 -- 172.21.15.16:0/2053598357 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e680034e0 con 0x7f9e800cacf0 2024-08-17T02:45:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.412+0000 7f9e85800640 1 -- 172.21.15.16:0/2053598357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e800ce6d0 msgr2=0x7f9e800ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.412+0000 7f9e85800640 1 --2- 172.21.15.16:0/2053598357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e800ce6d0 0x7f9e800ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.412+0000 7f9e85800640 1 -- 172.21.15.16:0/2053598357 >> v1:172.21.15.16:6789/0 conn(0x7f9e800cacf0 legacy=0x7f9e800c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.412+0000 7f9e85800640 1 -- 172.21.15.16:0/2053598357 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e800cb7a0 con 0x7f9e800aed50 2024-08-17T02:45:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.412+0000 7f9e84e00640 1 --2- 172.21.15.16:0/2053598357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e800ce6d0 0x7f9e800ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:24.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.413+0000 7f9e85800640 1 --2- 172.21.15.16:0/2053598357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e800aed50 0x7f9e800ce190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009850 tx=0x7f9e740090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f167b912dbb40240 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.413+0000 7f9e7fe00640 1 -- 172.21.15.16:0/2053598357 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7402ad80 con 0x7f9e800aed50 2024-08-17T02:45:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.413+0000 7f9e7fe00640 1 -- 172.21.15.16:0/2053598357 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e74032440 con 0x7f9e800aed50 2024-08-17T02:45:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.413+0000 7f9e7fe00640 1 -- 172.21.15.16:0/2053598357 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74032800 con 0x7f9e800aed50 2024-08-17T02:45:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.414+0000 7f9e86c00640 1 -- 172.21.15.16:0/2053598357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e800aed50 msgr2=0x7f9e800ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.414+0000 7f9e86c00640 1 --2- 172.21.15.16:0/2053598357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e800aed50 0x7f9e800ce190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009850 tx=0x7f9e740090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.414+0000 7f9e86c00640 1 -- 172.21.15.16:0/2053598357 shutdown_connections 2024-08-17T02:45:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.414+0000 7f9e86c00640 1 --2- 172.21.15.16:0/2053598357 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e800ce6d0 0x7f9e800ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.414+0000 7f9e86c00640 1 --2- 172.21.15.16:0/2053598357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e800aed50 0x7f9e800ce190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.414+0000 7f9e86c00640 1 -- 172.21.15.16:0/2053598357 >> 172.21.15.16:0/2053598357 conn(0x7f9e80098e30 msgr2=0x7f9e8006eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.414+0000 7f9e86c00640 1 -- 172.21.15.16:0/2053598357 shutdown_connections 2024-08-17T02:45:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.414+0000 7f9e86c00640 1 -- 172.21.15.16:0/2053598357 wait complete. 2024-08-17T02:45:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.415+0000 7f9e86c00640 1 Processor -- start 2024-08-17T02:45:24.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.415+0000 7f9e86c00640 1 -- start start 2024-08-17T02:45:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.416+0000 7f9e86c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e800aed50 0x7f9e800b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.416+0000 7f9e86c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e800cacf0 0x7f9e800c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.416+0000 7f9e86c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e800ce6d0 0x7f9e800c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.416+0000 7f9e86c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800c7010 con 0x7f9e800aed50 2024-08-17T02:45:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.416+0000 7f9e86c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800b2970 con 0x7f9e800cacf0 2024-08-17T02:45:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.416+0000 7f9e86c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e80142170 con 0x7f9e800ce6d0 2024-08-17T02:45:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.416+0000 7f9e85800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e800aed50 0x7f9e800b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.416+0000 7f9e85800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e800aed50 0x7f9e800b5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57840/0 (socket says 172.21.15.16:57840) 2024-08-17T02:45:24.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.416+0000 7f9e85800640 1 -- 172.21.15.16:0/4007410737 learned_addr learned my addr 172.21.15.16:0/4007410737 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e84e00640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e800cacf0 0x7f9e800c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e86200640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e800ce6d0 0x7f9e800c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e85800640 1 -- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e800ce6d0 msgr2=0x7f9e800c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e85800640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e800ce6d0 0x7f9e800c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e85800640 1 -- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e800cacf0 msgr2=0x7f9e800c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e85800640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e800cacf0 0x7f9e800c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e85800640 1 -- 172.21.15.16:0/4007410737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c009390 con 0x7f9e800aed50 2024-08-17T02:45:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e86200640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e800ce6d0 0x7f9e800c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:24.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e85800640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e800aed50 0x7f9e800b5300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009850 tx=0x7f9e74032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:24.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e7cc00640 1 -- 172.21.15.16:0/4007410737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74031830 con 0x7f9e800aed50 2024-08-17T02:45:24.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e7cc00640 1 -- 172.21.15.16:0/4007410737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e74031990 con 0x7f9e800aed50 2024-08-17T02:45:24.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e7cc00640 1 -- 172.21.15.16:0/4007410737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74031d70 con 0x7f9e800aed50 2024-08-17T02:45:24.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.417+0000 7f9e86c00640 1 -- 172.21.15.16:0/4007410737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e74009390 con 0x7f9e800aed50 2024-08-17T02:45:24.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.418+0000 7f9e86c00640 1 -- 172.21.15.16:0/4007410737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e800c99a0 con 0x7f9e800aed50 2024-08-17T02:45:24.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.418+0000 7f9e86c00640 1 -- 172.21.15.16:0/4007410737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e80071050 con 0x7f9e800aed50 2024-08-17T02:45:24.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.419+0000 7f9e7cc00640 1 -- 172.21.15.16:0/4007410737 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f9e7403e660 con 0x7f9e800aed50 2024-08-17T02:45:24.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.420+0000 7f9e7cc00640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9e60065150 0x7f9e60067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.420+0000 7f9e7cc00640 1 -- 172.21.15.16:0/4007410737 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9e740a2490 con 0x7f9e800aed50 2024-08-17T02:45:24.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.420+0000 7f9e84e00640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9e60065150 0x7f9e60067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.421+0000 7f9e84e00640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9e60065150 0x7f9e60067600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e800c81c0 tx=0x7f9e6c0090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:24.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.424+0000 7f9e7cc00640 1 -- 172.21.15.16:0/4007410737 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9e74070560 con 0x7f9e800aed50 2024-08-17T02:45:24.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.602+0000 7f9e86c00640 1 -- 172.21.15.16:0/4007410737 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e8006f080 con 0x7f9e60065150 2024-08-17T02:45:24.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.611+0000 7f9e7cc00640 1 -- 172.21.15.16:0/4007410737 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264688 (secure 0 0 0) 0x7f9e8006f080 con 0x7f9e60065150 2024-08-17T02:45:24.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:24.619 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-17T02:45:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.615+0000 7f9e86c00640 1 -- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9e60065150 msgr2=0x7f9e60067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.615+0000 7f9e86c00640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9e60065150 0x7f9e60067600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e800c81c0 tx=0x7f9e6c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.615+0000 7f9e86c00640 1 -- 172.21.15.16:0/4007410737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e800aed50 msgr2=0x7f9e800b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.615+0000 7f9e86c00640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e800aed50 0x7f9e800b5300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009850 tx=0x7f9e74032000 comp rx=0 tx=0).stop 2024-08-17T02:45:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.615+0000 7f9e86c00640 1 -- 172.21.15.16:0/4007410737 shutdown_connections 2024-08-17T02:45:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.615+0000 7f9e86c00640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9e60065150 0x7f9e60067600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.615+0000 7f9e86c00640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e800ce6d0 0x7f9e800c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.615+0000 7f9e86c00640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e800cacf0 0x7f9e800c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.615+0000 7f9e86c00640 1 --2- 172.21.15.16:0/4007410737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e800aed50 0x7f9e800b5300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.615+0000 7f9e86c00640 1 -- 172.21.15.16:0/4007410737 >> 172.21.15.16:0/4007410737 conn(0x7f9e80098e30 msgr2=0x7f9e801585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.615+0000 7f9e86c00640 1 -- 172.21.15.16:0/4007410737 shutdown_connections 2024-08-17T02:45:24.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.615+0000 7f9e86c00640 1 -- 172.21.15.16:0/4007410737 wait complete. 2024-08-17T02:45:24.631 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-08-17T02:45:24.236459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398428,"kb_used_data":2920,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752576292,"statfs":{"total":773094113280,"available":770638123008,"internally_reserved":0,"allocated":2990080,"data_stored":1787916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.215821"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048440+0000","last_change":"2024-08-17T02:45:20.044663+0000","last_active":"2024-08-17T02:45:20.048440+0000","last_peered":"2024-08-17T02:45:20.048440+0000","last_clean":"2024-08-17T02:45:20.048440+0000","last_became_active":"2024-08-17T02:45:20.044412+0000","last_became_peered":"2024-08-17T02:45:20.044412+0000","last_unstale":"2024-08-17T02:45:20.048440+0000","last_undegraded":"2024-08-17T02:45:20.048440+0000","last_fullsized":"2024-08-17T02:45:20.048440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:36:50.133729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.038551+0000","last_change":"2024-08-17T02:45:20.035833+0000","last_active":"2024-08-17T02:45:20.038551+0000","last_peered":"2024-08-17T02:45:20.038551+0000","last_clean":"2024-08-17T02:45:20.038551+0000","last_became_active":"2024-08-17T02:45:20.035557+0000","last_became_peered":"2024-08-17T02:45:20.035557+0000","last_unstale":"2024-08-17T02:45:20.038551+0000","last_undegraded":"2024-08-17T02:45:20.038551+0000","last_fullsized":"2024-08-17T02:45:20.038551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:16:04.128578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.031258+0000","last_change":"2024-08-17T02:45:20.027132+0000","last_active":"2024-08-17T02:45:20.031258+0000","last_peered":"2024-08-17T02:45:20.031258+0000","last_clean":"2024-08-17T02:45:20.031258+0000","last_became_active":"2024-08-17T02:45:20.026869+0000","last_became_peered":"2024-08-17T02:45:20.026869+0000","last_unstale":"2024-08-17T02:45:20.031258+0000","last_undegraded":"2024-08-17T02:45:20.031258+0000","last_fullsized":"2024-08-17T02:45:20.031258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:50:24.600922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.050217+0000","last_change":"2024-08-17T02:45:20.042821+0000","last_active":"2024-08-17T02:45:20.050217+0000","last_peered":"2024-08-17T02:45:20.050217+0000","last_clean":"2024-08-17T02:45:20.050217+0000","last_became_active":"2024-08-17T02:45:20.042556+0000","last_became_peered":"2024-08-17T02:45:20.042556+0000","last_unstale":"2024-08-17T02:45:20.050217+0000","last_undegraded":"2024-08-17T02:45:20.050217+0000","last_fullsized":"2024-08-17T02:45:20.050217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:04:44.206575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.034352+0000","last_change":"2024-08-17T02:45:20.030193+0000","last_active":"2024-08-17T02:45:20.034352+0000","last_peered":"2024-08-17T02:45:20.034352+0000","last_clean":"2024-08-17T02:45:20.034352+0000","last_became_active":"2024-08-17T02:45:20.029651+0000","last_became_peered":"2024-08-17T02:45:20.029651+0000","last_unstale":"2024-08-17T02:45:20.034352+0000","last_undegraded":"2024-08-17T02:45:20.034352+0000","last_fullsized":"2024-08-17T02:45:20.034352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:09:19.783891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047891+0000","last_change":"2024-08-17T02:45:20.040335+0000","last_active":"2024-08-17T02:45:20.047891+0000","last_peered":"2024-08-17T02:45:20.047891+0000","last_clean":"2024-08-17T02:45:20.047891+0000","last_became_active":"2024-08-17T02:45:20.039917+0000","last_became_peered":"2024-08-17T02:45:20.039917+0000","last_unstale":"2024-08-17T02:45:20.047891+0000","last_undegraded":"2024-08-17T02:45:20.047891+0000","last_fullsized":"2024-08-17T02:45:20.047891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:08.644038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047626+0000","last_change":"2024-08-17T02:45:20.036503+0000","last_active":"2024-08-17T02:45:20.047626+0000","last_peered":"2024-08-17T02:45:20.047626+0000","last_clean":"2024-08-17T02:45:20.047626+0000","last_became_active":"2024-08-17T02:45:20.033814+0000","last_became_peered":"2024-08-17T02:45:20.033814+0000","last_unstale":"2024-08-17T02:45:20.047626+0000","last_undegraded":"2024-08-17T02:45:20.047626+0000","last_fullsized":"2024-08-17T02:45:20.047626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:32:46.909589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051312+0000","last_change":"2024-08-17T02:45:20.046161+0000","last_active":"2024-08-17T02:45:20.051312+0000","last_peered":"2024-08-17T02:45:20.051312+0000","last_clean":"2024-08-17T02:45:20.051312+0000","last_became_active":"2024-08-17T02:45:20.042204+0000","last_became_peered":"2024-08-17T02:45:20.042204+0000","last_unstale":"2024-08-17T02:45:20.051312+0000","last_undegraded":"2024-08-17T02:45:20.051312+0000","last_fullsized":"2024-08-17T02:45:20.051312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:32:46.909589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.021896+0000","last_change":"2024-08-17T02:45:20.019114+0000","last_active":"2024-08-17T02:45:20.021896+0000","last_peered":"2024-08-17T02:45:20.021896+0000","last_clean":"2024-08-17T02:45:20.021896+0000","last_became_active":"2024-08-17T02:45:20.018864+0000","last_became_peered":"2024-08-17T02:45:20.018864+0000","last_unstale":"2024-08-17T02:45:20.021896+0000","last_undegraded":"2024-08-17T02:45:20.021896+0000","last_fullsized":"2024-08-17T02:45:20.021896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:05:58.485958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048643+0000","last_change":"2024-08-17T02:45:20.044934+0000","last_active":"2024-08-17T02:45:20.048643+0000","last_peered":"2024-08-17T02:45:20.048643+0000","last_clean":"2024-08-17T02:45:20.048643+0000","last_became_active":"2024-08-17T02:45:20.044408+0000","last_became_peered":"2024-08-17T02:45:20.044408+0000","last_unstale":"2024-08-17T02:45:20.048643+0000","last_undegraded":"2024-08-17T02:45:20.048643+0000","last_fullsized":"2024-08-17T02:45:20.048643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:50:24.600922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.049471+0000","last_change":"2024-08-17T02:45:20.044628+0000","last_active":"2024-08-17T02:45:20.049471+0000","last_peered":"2024-08-17T02:45:20.049471+0000","last_clean":"2024-08-17T02:45:20.049471+0000","last_became_active":"2024-08-17T02:45:20.044329+0000","last_became_peered":"2024-08-17T02:45:20.044329+0000","last_unstale":"2024-08-17T02:45:20.049471+0000","last_undegraded":"2024-08-17T02:45:20.049471+0000","last_fullsized":"2024-08-17T02:45:20.049471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:31.312177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048512+0000","last_change":"2024-08-17T02:45:20.042172+0000","last_active":"2024-08-17T02:45:20.048512+0000","last_peered":"2024-08-17T02:45:20.048512+0000","last_clean":"2024-08-17T02:45:20.048512+0000","last_became_active":"2024-08-17T02:45:20.041703+0000","last_became_peered":"2024-08-17T02:45:20.041703+0000","last_unstale":"2024-08-17T02:45:20.048512+0000","last_undegraded":"2024-08-17T02:45:20.048512+0000","last_fullsized":"2024-08-17T02:45:20.048512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:42.062952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052573+0000","last_change":"2024-08-17T02:45:20.045385+0000","last_active":"2024-08-17T02:45:20.052573+0000","last_peered":"2024-08-17T02:45:20.052573+0000","last_clean":"2024-08-17T02:45:20.052573+0000","last_became_active":"2024-08-17T02:45:20.045105+0000","last_became_peered":"2024-08-17T02:45:20.045105+0000","last_unstale":"2024-08-17T02:45:20.052573+0000","last_undegraded":"2024-08-17T02:45:20.052573+0000","last_fullsized":"2024-08-17T02:45:20.052573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:33:18.470465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051658+0000","last_change":"2024-08-17T02:45:20.039999+0000","last_active":"2024-08-17T02:45:20.051658+0000","last_peered":"2024-08-17T02:45:20.051658+0000","last_clean":"2024-08-17T02:45:20.051658+0000","last_became_active":"2024-08-17T02:45:20.039656+0000","last_became_peered":"2024-08-17T02:45:20.039656+0000","last_unstale":"2024-08-17T02:45:20.051658+0000","last_undegraded":"2024-08-17T02:45:20.051658+0000","last_fullsized":"2024-08-17T02:45:20.051658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:02:17.358927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.049636+0000","last_change":"2024-08-17T02:45:20.042360+0000","last_active":"2024-08-17T02:45:20.049636+0000","last_peered":"2024-08-17T02:45:20.049636+0000","last_clean":"2024-08-17T02:45:20.049636+0000","last_became_active":"2024-08-17T02:45:20.042078+0000","last_became_peered":"2024-08-17T02:45:20.042078+0000","last_unstale":"2024-08-17T02:45:20.049636+0000","last_undegraded":"2024-08-17T02:45:20.049636+0000","last_fullsized":"2024-08-17T02:45:20.049636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:41:49.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040453+0000","last_change":"2024-08-17T02:45:20.026340+0000","last_active":"2024-08-17T02:45:20.040453+0000","last_peered":"2024-08-17T02:45:20.040453+0000","last_clean":"2024-08-17T02:45:20.040453+0000","last_became_active":"2024-08-17T02:45:20.026045+0000","last_became_peered":"2024-08-17T02:45:20.026045+0000","last_unstale":"2024-08-17T02:45:20.040453+0000","last_undegraded":"2024-08-17T02:45:20.040453+0000","last_fullsized":"2024-08-17T02:45:20.040453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:15:44.912061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052119+0000","last_change":"2024-08-17T02:45:20.037915+0000","last_active":"2024-08-17T02:45:20.052119+0000","last_peered":"2024-08-17T02:45:20.052119+0000","last_clean":"2024-08-17T02:45:20.052119+0000","last_became_active":"2024-08-17T02:45:20.034495+0000","last_became_peered":"2024-08-17T02:45:20.034495+0000","last_unstale":"2024-08-17T02:45:20.052119+0000","last_undegraded":"2024-08-17T02:45:20.052119+0000","last_fullsized":"2024-08-17T02:45:20.052119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:37:11.925388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.056304+0000","last_change":"2024-08-17T02:45:20.054620+0000","last_active":"2024-08-17T02:45:20.056304+0000","last_peered":"2024-08-17T02:45:20.056304+0000","last_clean":"2024-08-17T02:45:20.056304+0000","last_became_active":"2024-08-17T02:45:20.054100+0000","last_became_peered":"2024-08-17T02:45:20.054100+0000","last_unstale":"2024-08-17T02:45:20.056304+0000","last_undegraded":"2024-08-17T02:45:20.056304+0000","last_fullsized":"2024-08-17T02:45:20.056304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:08.644038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.041022+0000","last_change":"2024-08-17T02:45:20.031250+0000","last_active":"2024-08-17T02:45:20.041022+0000","last_peered":"2024-08-17T02:45:20.041022+0000","last_clean":"2024-08-17T02:45:20.041022+0000","last_became_active":"2024-08-17T02:45:20.026261+0000","last_became_peered":"2024-08-17T02:45:20.026261+0000","last_unstale":"2024-08-17T02:45:20.041022+0000","last_undegraded":"2024-08-17T02:45:20.041022+0000","last_fullsized":"2024-08-17T02:45:20.041022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:54:59.143806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048503+0000","last_change":"2024-08-17T02:45:20.041999+0000","last_active":"2024-08-17T02:45:20.048503+0000","last_peered":"2024-08-17T02:45:20.048503+0000","last_clean":"2024-08-17T02:45:20.048503+0000","last_became_active":"2024-08-17T02:45:20.041438+0000","last_became_peered":"2024-08-17T02:45:20.041438+0000","last_unstale":"2024-08-17T02:45:20.048503+0000","last_undegraded":"2024-08-17T02:45:20.048503+0000","last_fullsized":"2024-08-17T02:45:20.048503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:20.121506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.032037+0000","last_change":"2024-08-17T02:45:20.027927+0000","last_active":"2024-08-17T02:45:20.032037+0000","last_peered":"2024-08-17T02:45:20.032037+0000","last_clean":"2024-08-17T02:45:20.032037+0000","last_became_active":"2024-08-17T02:45:20.026932+0000","last_became_peered":"2024-08-17T02:45:20.026932+0000","last_unstale":"2024-08-17T02:45:20.032037+0000","last_undegraded":"2024-08-17T02:45:20.032037+0000","last_fullsized":"2024-08-17T02:45:20.032037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:33:12.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.032654+0000","last_change":"2024-08-17T02:45:20.031118+0000","last_active":"2024-08-17T02:45:20.032654+0000","last_peered":"2024-08-17T02:45:20.032654+0000","last_clean":"2024-08-17T02:45:20.032654+0000","last_became_active":"2024-08-17T02:45:20.030216+0000","last_became_peered":"2024-08-17T02:45:20.030216+0000","last_unstale":"2024-08-17T02:45:20.032654+0000","last_undegraded":"2024-08-17T02:45:20.032654+0000","last_fullsized":"2024-08-17T02:45:20.032654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:41:33.774280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051361+0000","last_change":"2024-08-17T02:45:20.045808+0000","last_active":"2024-08-17T02:45:20.051361+0000","last_peered":"2024-08-17T02:45:20.051361+0000","last_clean":"2024-08-17T02:45:20.051361+0000","last_became_active":"2024-08-17T02:45:20.045029+0000","last_became_peered":"2024-08-17T02:45:20.045029+0000","last_unstale":"2024-08-17T02:45:20.051361+0000","last_undegraded":"2024-08-17T02:45:20.051361+0000","last_fullsized":"2024-08-17T02:45:20.051361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:30.359495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040472+0000","last_change":"2024-08-17T02:45:20.028434+0000","last_active":"2024-08-17T02:45:20.040472+0000","last_peered":"2024-08-17T02:45:20.040472+0000","last_clean":"2024-08-17T02:45:20.040472+0000","last_became_active":"2024-08-17T02:45:20.026769+0000","last_became_peered":"2024-08-17T02:45:20.026769+0000","last_unstale":"2024-08-17T02:45:20.040472+0000","last_undegraded":"2024-08-17T02:45:20.040472+0000","last_fullsized":"2024-08-17T02:45:20.040472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:40:57.163688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051347+0000","last_change":"2024-08-17T02:45:20.046140+0000","last_active":"2024-08-17T02:45:20.051347+0000","last_peered":"2024-08-17T02:45:20.051347+0000","last_clean":"2024-08-17T02:45:20.051347+0000","last_became_active":"2024-08-17T02:45:20.043780+0000","last_became_peered":"2024-08-17T02:45:20.043780+0000","last_unstale":"2024-08-17T02:45:20.051347+0000","last_undegraded":"2024-08-17T02:45:20.051347+0000","last_fullsized":"2024-08-17T02:45:20.051347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:15:44.912061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047741+0000","last_change":"2024-08-17T02:45:20.039484+0000","last_active":"2024-08-17T02:45:20.047741+0000","last_peered":"2024-08-17T02:45:20.047741+0000","last_clean":"2024-08-17T02:45:20.047741+0000","last_became_active":"2024-08-17T02:45:20.039202+0000","last_became_peered":"2024-08-17T02:45:20.039202+0000","last_unstale":"2024-08-17T02:45:20.047741+0000","last_undegraded":"2024-08-17T02:45:20.047741+0000","last_fullsized":"2024-08-17T02:45:20.047741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:37:57.900723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051555+0000","last_change":"2024-08-17T02:45:20.039636+0000","last_active":"2024-08-17T02:45:20.051555+0000","last_peered":"2024-08-17T02:45:20.051555+0000","last_clean":"2024-08-17T02:45:20.051555+0000","last_became_active":"2024-08-17T02:45:20.039357+0000","last_became_peered":"2024-08-17T02:45:20.039357+0000","last_unstale":"2024-08-17T02:45:20.051555+0000","last_undegraded":"2024-08-17T02:45:20.051555+0000","last_fullsized":"2024-08-17T02:45:20.051555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:53:40.660806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052416+0000","last_change":"2024-08-17T02:45:20.044576+0000","last_active":"2024-08-17T02:45:20.052416+0000","last_peered":"2024-08-17T02:45:20.052416+0000","last_clean":"2024-08-17T02:45:20.052416+0000","last_became_active":"2024-08-17T02:45:20.044338+0000","last_became_peered":"2024-08-17T02:45:20.044338+0000","last_unstale":"2024-08-17T02:45:20.052416+0000","last_undegraded":"2024-08-17T02:45:20.052416+0000","last_fullsized":"2024-08-17T02:45:20.052416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:33:12.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.049261+0000","last_change":"2024-08-17T02:45:20.041424+0000","last_active":"2024-08-17T02:45:20.049261+0000","last_peered":"2024-08-17T02:45:20.049261+0000","last_clean":"2024-08-17T02:45:20.049261+0000","last_became_active":"2024-08-17T02:45:20.041158+0000","last_became_peered":"2024-08-17T02:45:20.041158+0000","last_unstale":"2024-08-17T02:45:20.049261+0000","last_undegraded":"2024-08-17T02:45:20.049261+0000","last_fullsized":"2024-08-17T02:45:20.049261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:09:19.783891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.041640+0000","last_change":"2024-08-17T02:45:20.027101+0000","last_active":"2024-08-17T02:45:20.041640+0000","last_peered":"2024-08-17T02:45:20.041640+0000","last_clean":"2024-08-17T02:45:20.041640+0000","last_became_active":"2024-08-17T02:45:20.026792+0000","last_became_peered":"2024-08-17T02:45:20.026792+0000","last_unstale":"2024-08-17T02:45:20.041640+0000","last_undegraded":"2024-08-17T02:45:20.041640+0000","last_fullsized":"2024-08-17T02:45:20.041640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:27:01.489949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047767+0000","last_change":"2024-08-17T02:45:20.044023+0000","last_active":"2024-08-17T02:45:20.047767+0000","last_peered":"2024-08-17T02:45:20.047767+0000","last_clean":"2024-08-17T02:45:20.047767+0000","last_became_active":"2024-08-17T02:45:20.043731+0000","last_became_peered":"2024-08-17T02:45:20.043731+0000","last_unstale":"2024-08-17T02:45:20.047767+0000","last_undegraded":"2024-08-17T02:45:20.047767+0000","last_fullsized":"2024-08-17T02:45:20.047767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:41:33.774280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048891+0000","last_change":"2024-08-17T02:45:20.042822+0000","last_active":"2024-08-17T02:45:20.048891+0000","last_peered":"2024-08-17T02:45:20.048891+0000","last_clean":"2024-08-17T02:45:20.048891+0000","last_became_active":"2024-08-17T02:45:20.042501+0000","last_became_peered":"2024-08-17T02:45:20.042501+0000","last_unstale":"2024-08-17T02:45:20.048891+0000","last_undegraded":"2024-08-17T02:45:20.048891+0000","last_fullsized":"2024-08-17T02:45:20.048891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:53:40.660806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040933+0000","last_change":"2024-08-17T02:45:20.032549+0000","last_active":"2024-08-17T02:45:20.040933+0000","last_peered":"2024-08-17T02:45:20.040933+0000","last_clean":"2024-08-17T02:45:20.040933+0000","last_became_active":"2024-08-17T02:45:20.032196+0000","last_became_peered":"2024-08-17T02:45:20.032196+0000","last_unstale":"2024-08-17T02:45:20.040933+0000","last_undegraded":"2024-08-17T02:45:20.040933+0000","last_fullsized":"2024-08-17T02:45:20.040933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:08.644038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.053812+0000","last_change":"2024-08-17T02:45:19.040955+0000","last_active":"2024-08-17T02:45:20.053812+0000","last_peered":"2024-08-17T02:45:20.053812+0000","last_clean":"2024-08-17T02:45:20.053812+0000","last_became_active":"2024-08-17T02:45:19.039340+0000","last_became_peered":"2024-08-17T02:45:19.039340+0000","last_unstale":"2024-08-17T02:45:20.053812+0000","last_undegraded":"2024-08-17T02:45:20.053812+0000","last_fullsized":"2024-08-17T02:45:20.053812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040815+0000","last_change":"2024-08-17T02:45:20.035907+0000","last_active":"2024-08-17T02:45:20.040815+0000","last_peered":"2024-08-17T02:45:20.040815+0000","last_clean":"2024-08-17T02:45:20.040815+0000","last_became_active":"2024-08-17T02:45:20.035670+0000","last_became_peered":"2024-08-17T02:45:20.035670+0000","last_unstale":"2024-08-17T02:45:20.040815+0000","last_undegraded":"2024-08-17T02:45:20.040815+0000","last_fullsized":"2024-08-17T02:45:20.040815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:41:49.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054523+0000","last_change":"2024-08-17T02:45:19.048167+0000","last_active":"2024-08-17T02:45:20.054523+0000","last_peered":"2024-08-17T02:45:20.054523+0000","last_clean":"2024-08-17T02:45:20.054523+0000","last_became_active":"2024-08-17T02:45:19.047739+0000","last_became_peered":"2024-08-17T02:45:19.047739+0000","last_unstale":"2024-08-17T02:45:20.054523+0000","last_undegraded":"2024-08-17T02:45:20.054523+0000","last_fullsized":"2024-08-17T02:45:20.054523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:33:17.468626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.033343+0000","last_change":"2024-08-17T02:45:19.043960+0000","last_active":"2024-08-17T02:45:20.033343+0000","last_peered":"2024-08-17T02:45:20.033343+0000","last_clean":"2024-08-17T02:45:20.033343+0000","last_became_active":"2024-08-17T02:45:19.042684+0000","last_became_peered":"2024-08-17T02:45:19.042684+0000","last_unstale":"2024-08-17T02:45:20.033343+0000","last_undegraded":"2024-08-17T02:45:20.033343+0000","last_fullsized":"2024-08-17T02:45:20.033343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.038584+0000","last_change":"2024-08-17T02:45:20.034766+0000","last_active":"2024-08-17T02:45:20.038584+0000","last_peered":"2024-08-17T02:45:20.038584+0000","last_clean":"2024-08-17T02:45:20.038584+0000","last_became_active":"2024-08-17T02:45:20.034450+0000","last_became_peered":"2024-08-17T02:45:20.034450+0000","last_unstale":"2024-08-17T02:45:20.038584+0000","last_undegraded":"2024-08-17T02:45:20.038584+0000","last_fullsized":"2024-08-17T02:45:20.038584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:02:55.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.013694+0000","last_change":"2024-08-17T02:45:19.050133+0000","last_active":"2024-08-17T02:45:20.013694+0000","last_peered":"2024-08-17T02:45:20.013694+0000","last_clean":"2024-08-17T02:45:20.013694+0000","last_became_active":"2024-08-17T02:45:19.049873+0000","last_became_peered":"2024-08-17T02:45:19.049873+0000","last_unstale":"2024-08-17T02:45:20.013694+0000","last_undegraded":"2024-08-17T02:45:20.013694+0000","last_fullsized":"2024-08-17T02:45:20.013694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:05:57.484119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.031101+0000","last_change":"2024-08-17T02:45:20.023134+0000","last_active":"2024-08-17T02:45:20.031101+0000","last_peered":"2024-08-17T02:45:20.031101+0000","last_clean":"2024-08-17T02:45:20.031101+0000","last_became_active":"2024-08-17T02:45:20.022844+0000","last_became_peered":"2024-08-17T02:45:20.022844+0000","last_unstale":"2024-08-17T02:45:20.031101+0000","last_undegraded":"2024-08-17T02:45:20.031101+0000","last_fullsized":"2024-08-17T02:45:20.031101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:30.359495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.018045+0000","last_change":"2024-08-17T02:45:19.046922+0000","last_active":"2024-08-17T02:45:20.018045+0000","last_peered":"2024-08-17T02:45:20.018045+0000","last_clean":"2024-08-17T02:45:20.018045+0000","last_became_active":"2024-08-17T02:45:19.046578+0000","last_became_peered":"2024-08-17T02:45:19.046578+0000","last_unstale":"2024-08-17T02:45:20.018045+0000","last_undegraded":"2024-08-17T02:45:20.018045+0000","last_fullsized":"2024-08-17T02:45:20.018045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047730+0000","last_change":"2024-08-17T02:45:20.039463+0000","last_active":"2024-08-17T02:45:20.047730+0000","last_peered":"2024-08-17T02:45:20.047730+0000","last_clean":"2024-08-17T02:45:20.047730+0000","last_became_active":"2024-08-17T02:45:20.038869+0000","last_became_peered":"2024-08-17T02:45:20.038869+0000","last_unstale":"2024-08-17T02:45:20.047730+0000","last_undegraded":"2024-08-17T02:45:20.047730+0000","last_fullsized":"2024-08-17T02:45:20.047730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:33:18.470465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.035076+0000","last_change":"2024-08-17T02:45:19.048740+0000","last_active":"2024-08-17T02:45:20.035076+0000","last_peered":"2024-08-17T02:45:20.035076+0000","last_clean":"2024-08-17T02:45:20.035076+0000","last_became_active":"2024-08-17T02:45:19.047688+0000","last_became_peered":"2024-08-17T02:45:19.047688+0000","last_unstale":"2024-08-17T02:45:20.035076+0000","last_undegraded":"2024-08-17T02:45:20.035076+0000","last_fullsized":"2024-08-17T02:45:20.035076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:43.862231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051327+0000","last_change":"2024-08-17T02:45:20.046160+0000","last_active":"2024-08-17T02:45:20.051327+0000","last_peered":"2024-08-17T02:45:20.051327+0000","last_clean":"2024-08-17T02:45:20.051327+0000","last_became_active":"2024-08-17T02:45:20.045893+0000","last_became_peered":"2024-08-17T02:45:20.045893+0000","last_unstale":"2024-08-17T02:45:20.051327+0000","last_undegraded":"2024-08-17T02:45:20.051327+0000","last_fullsized":"2024-08-17T02:45:20.051327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:03:07.944484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054309+0000","last_change":"2024-08-17T02:45:19.043386+0000","last_active":"2024-08-17T02:45:20.054309+0000","last_peered":"2024-08-17T02:45:20.054309+0000","last_clean":"2024-08-17T02:45:20.054309+0000","last_became_active":"2024-08-17T02:45:19.043050+0000","last_became_peered":"2024-08-17T02:45:19.043050+0000","last_unstale":"2024-08-17T02:45:20.054309+0000","last_undegraded":"2024-08-17T02:45:20.054309+0000","last_fullsized":"2024-08-17T02:45:20.054309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.042043+0000","last_change":"2024-08-17T02:45:20.029004+0000","last_active":"2024-08-17T02:45:20.042043+0000","last_peered":"2024-08-17T02:45:20.042043+0000","last_clean":"2024-08-17T02:45:20.042043+0000","last_became_active":"2024-08-17T02:45:20.028698+0000","last_became_peered":"2024-08-17T02:45:20.028698+0000","last_unstale":"2024-08-17T02:45:20.042043+0000","last_undegraded":"2024-08-17T02:45:20.042043+0000","last_fullsized":"2024-08-17T02:45:20.042043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:42.062952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.058174+0000","last_change":"2024-08-17T02:45:19.034956+0000","last_active":"2024-08-17T02:45:20.058174+0000","last_peered":"2024-08-17T02:45:20.058174+0000","last_clean":"2024-08-17T02:45:20.058174+0000","last_became_active":"2024-08-17T02:45:19.034658+0000","last_became_peered":"2024-08-17T02:45:19.034658+0000","last_unstale":"2024-08-17T02:45:20.058174+0000","last_undegraded":"2024-08-17T02:45:20.058174+0000","last_fullsized":"2024-08-17T02:45:20.058174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048217+0000","last_change":"2024-08-17T02:45:20.039165+0000","last_active":"2024-08-17T02:45:20.048217+0000","last_peered":"2024-08-17T02:45:20.048217+0000","last_clean":"2024-08-17T02:45:20.048217+0000","last_became_active":"2024-08-17T02:45:20.038923+0000","last_became_peered":"2024-08-17T02:45:20.038923+0000","last_unstale":"2024-08-17T02:45:20.048217+0000","last_undegraded":"2024-08-17T02:45:20.048217+0000","last_fullsized":"2024-08-17T02:45:20.048217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:03:07.944484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.053604+0000","last_change":"2024-08-17T02:45:19.044788+0000","last_active":"2024-08-17T02:45:20.053604+0000","last_peered":"2024-08-17T02:45:20.053604+0000","last_clean":"2024-08-17T02:45:20.053604+0000","last_became_active":"2024-08-17T02:45:19.044538+0000","last_became_peered":"2024-08-17T02:45:19.044538+0000","last_unstale":"2024-08-17T02:45:20.053604+0000","last_undegraded":"2024-08-17T02:45:20.053604+0000","last_fullsized":"2024-08-17T02:45:20.053604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:44:52.129209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052358+0000","last_change":"2024-08-17T02:45:20.043190+0000","last_active":"2024-08-17T02:45:20.052358+0000","last_peered":"2024-08-17T02:45:20.052358+0000","last_clean":"2024-08-17T02:45:20.052358+0000","last_became_active":"2024-08-17T02:45:20.042860+0000","last_became_peered":"2024-08-17T02:45:20.042860+0000","last_unstale":"2024-08-17T02:45:20.052358+0000","last_undegraded":"2024-08-17T02:45:20.052358+0000","last_fullsized":"2024-08-17T02:45:20.052358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:55:18.896035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.013972+0000","last_change":"2024-08-17T02:45:19.047294+0000","last_active":"2024-08-17T02:45:20.013972+0000","last_peered":"2024-08-17T02:45:20.013972+0000","last_clean":"2024-08-17T02:45:20.013972+0000","last_became_active":"2024-08-17T02:45:19.042180+0000","last_became_peered":"2024-08-17T02:45:19.042180+0000","last_unstale":"2024-08-17T02:45:20.013972+0000","last_undegraded":"2024-08-17T02:45:20.013972+0000","last_fullsized":"2024-08-17T02:45:20.013972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:37:56.898883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052410+0000","last_change":"2024-08-17T02:45:20.038226+0000","last_active":"2024-08-17T02:45:20.052410+0000","last_peered":"2024-08-17T02:45:20.052410+0000","last_clean":"2024-08-17T02:45:20.052410+0000","last_became_active":"2024-08-17T02:45:20.034659+0000","last_became_peered":"2024-08-17T02:45:20.034659+0000","last_unstale":"2024-08-17T02:45:20.052410+0000","last_undegraded":"2024-08-17T02:45:20.052410+0000","last_fullsized":"2024-08-17T02:45:20.052410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:27:53.329893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.036680+0000","last_change":"2024-08-17T02:45:19.052673+0000","last_active":"2024-08-17T02:45:20.036680+0000","last_peered":"2024-08-17T02:45:20.036680+0000","last_clean":"2024-08-17T02:45:20.036680+0000","last_became_active":"2024-08-17T02:45:19.052404+0000","last_became_peered":"2024-08-17T02:45:19.052404+0000","last_unstale":"2024-08-17T02:45:20.036680+0000","last_undegraded":"2024-08-17T02:45:20.036680+0000","last_fullsized":"2024-08-17T02:45:20.036680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:46:09.850664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.039882+0000","last_change":"2024-08-17T02:45:20.031377+0000","last_active":"2024-08-17T02:45:20.039882+0000","last_peered":"2024-08-17T02:45:20.039882+0000","last_clean":"2024-08-17T02:45:20.039882+0000","last_became_active":"2024-08-17T02:45:20.031103+0000","last_became_peered":"2024-08-17T02:45:20.031103+0000","last_unstale":"2024-08-17T02:45:20.039882+0000","last_undegraded":"2024-08-17T02:45:20.039882+0000","last_fullsized":"2024-08-17T02:45:20.039882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:04:44.206575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054426+0000","last_change":"2024-08-17T02:45:19.042568+0000","last_active":"2024-08-17T02:45:20.054426+0000","last_peered":"2024-08-17T02:45:20.054426+0000","last_clean":"2024-08-17T02:45:20.054426+0000","last_became_active":"2024-08-17T02:45:19.041846+0000","last_became_peered":"2024-08-17T02:45:19.041846+0000","last_unstale":"2024-08-17T02:45:20.054426+0000","last_undegraded":"2024-08-17T02:45:20.054426+0000","last_fullsized":"2024-08-17T02:45:20.054426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.042968+0000","last_change":"2024-08-17T02:45:20.029516+0000","last_active":"2024-08-17T02:45:20.042968+0000","last_peered":"2024-08-17T02:45:20.042968+0000","last_clean":"2024-08-17T02:45:20.042968+0000","last_became_active":"2024-08-17T02:45:20.027167+0000","last_became_peered":"2024-08-17T02:45:20.027167+0000","last_unstale":"2024-08-17T02:45:20.042968+0000","last_undegraded":"2024-08-17T02:45:20.042968+0000","last_fullsized":"2024-08-17T02:45:20.042968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:20.121506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.056082+0000","last_change":"2024-08-17T02:45:19.038843+0000","last_active":"2024-08-17T02:45:20.056082+0000","last_peered":"2024-08-17T02:45:20.056082+0000","last_clean":"2024-08-17T02:45:20.056082+0000","last_became_active":"2024-08-17T02:45:19.038608+0000","last_became_peered":"2024-08-17T02:45:19.038608+0000","last_unstale":"2024-08-17T02:45:20.056082+0000","last_undegraded":"2024-08-17T02:45:20.056082+0000","last_fullsized":"2024-08-17T02:45:20.056082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:25:12.592566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047411+0000","last_change":"2024-08-17T02:45:20.037709+0000","last_active":"2024-08-17T02:45:20.047411+0000","last_peered":"2024-08-17T02:45:20.047411+0000","last_clean":"2024-08-17T02:45:20.047411+0000","last_became_active":"2024-08-17T02:45:20.037461+0000","last_became_peered":"2024-08-17T02:45:20.037461+0000","last_unstale":"2024-08-17T02:45:20.047411+0000","last_undegraded":"2024-08-17T02:45:20.047411+0000","last_fullsized":"2024-08-17T02:45:20.047411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:05:58.485958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051949+0000","last_change":"2024-08-17T02:45:19.039018+0000","last_active":"2024-08-17T02:45:20.051949+0000","last_peered":"2024-08-17T02:45:20.051949+0000","last_clean":"2024-08-17T02:45:20.051949+0000","last_became_active":"2024-08-17T02:45:19.036850+0000","last_became_peered":"2024-08-17T02:45:19.036850+0000","last_unstale":"2024-08-17T02:45:20.051949+0000","last_undegraded":"2024-08-17T02:45:20.051949+0000","last_fullsized":"2024-08-17T02:45:20.051949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.033842+0000","last_change":"2024-08-17T02:45:20.030655+0000","last_active":"2024-08-17T02:45:20.033842+0000","last_peered":"2024-08-17T02:45:20.033842+0000","last_clean":"2024-08-17T02:45:20.033842+0000","last_became_active":"2024-08-17T02:45:20.030439+0000","last_became_peered":"2024-08-17T02:45:20.030439+0000","last_unstale":"2024-08-17T02:45:20.033842+0000","last_undegraded":"2024-08-17T02:45:20.033842+0000","last_fullsized":"2024-08-17T02:45:20.033842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:15:44.912061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.055739+0000","last_change":"2024-08-17T02:45:19.045933+0000","last_active":"2024-08-17T02:45:20.055739+0000","last_peered":"2024-08-17T02:45:20.055739+0000","last_clean":"2024-08-17T02:45:20.055739+0000","last_became_active":"2024-08-17T02:45:19.045649+0000","last_became_peered":"2024-08-17T02:45:19.045649+0000","last_unstale":"2024-08-17T02:45:20.055739+0000","last_undegraded":"2024-08-17T02:45:20.055739+0000","last_fullsized":"2024-08-17T02:45:20.055739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048520+0000","last_change":"2024-08-17T02:45:20.043745+0000","last_active":"2024-08-17T02:45:20.048520+0000","last_peered":"2024-08-17T02:45:20.048520+0000","last_clean":"2024-08-17T02:45:20.048520+0000","last_became_active":"2024-08-17T02:45:20.043470+0000","last_became_peered":"2024-08-17T02:45:20.043470+0000","last_unstale":"2024-08-17T02:45:20.048520+0000","last_undegraded":"2024-08-17T02:45:20.048520+0000","last_fullsized":"2024-08-17T02:45:20.048520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:00:06.622218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.033701+0000","last_change":"2024-08-17T02:45:19.040219+0000","last_active":"2024-08-17T02:45:20.033701+0000","last_peered":"2024-08-17T02:45:20.033701+0000","last_clean":"2024-08-17T02:45:20.033701+0000","last_became_active":"2024-08-17T02:45:19.039816+0000","last_became_peered":"2024-08-17T02:45:19.039816+0000","last_unstale":"2024-08-17T02:45:20.033701+0000","last_undegraded":"2024-08-17T02:45:20.033701+0000","last_fullsized":"2024-08-17T02:45:20.033701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040997+0000","last_change":"2024-08-17T02:45:20.027155+0000","last_active":"2024-08-17T02:45:20.040997+0000","last_peered":"2024-08-17T02:45:20.040997+0000","last_clean":"2024-08-17T02:45:20.040997+0000","last_became_active":"2024-08-17T02:45:20.026102+0000","last_became_peered":"2024-08-17T02:45:20.026102+0000","last_unstale":"2024-08-17T02:45:20.040997+0000","last_undegraded":"2024-08-17T02:45:20.040997+0000","last_fullsized":"2024-08-17T02:45:20.040997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:05:05.162709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.017763+0000","last_change":"2024-08-17T02:45:16.001117+0000","last_active":"2024-08-17T02:45:20.017763+0000","last_peered":"2024-08-17T02:45:20.017763+0000","last_clean":"2024-08-17T02:45:20.017763+0000","last_became_active":"2024-08-17T02:45:16.000788+0000","last_became_peered":"2024-08-17T02:45:16.000788+0000","last_unstale":"2024-08-17T02:45:20.017763+0000","last_undegraded":"2024-08-17T02:45:20.017763+0000","last_fullsized":"2024-08-17T02:45:20.017763+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_clean_scrub_stamp":"2024-08-17T02:45:14.986707+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:27:37.295374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.014889+0000","last_change":"2024-08-17T02:45:19.042366+0000","last_active":"2024-08-17T02:45:20.014889+0000","last_peered":"2024-08-17T02:45:20.014889+0000","last_clean":"2024-08-17T02:45:20.014889+0000","last_became_active":"2024-08-17T02:45:19.039118+0000","last_became_peered":"2024-08-17T02:45:19.039118+0000","last_unstale":"2024-08-17T02:45:20.014889+0000","last_undegraded":"2024-08-17T02:45:20.014889+0000","last_fullsized":"2024-08-17T02:45:20.014889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:07.642199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.031922+0000","last_change":"2024-08-17T02:45:20.026830+0000","last_active":"2024-08-17T02:45:20.031922+0000","last_peered":"2024-08-17T02:45:20.031922+0000","last_clean":"2024-08-17T02:45:20.031922+0000","last_became_active":"2024-08-17T02:45:20.026135+0000","last_became_peered":"2024-08-17T02:45:20.026135+0000","last_unstale":"2024-08-17T02:45:20.031922+0000","last_undegraded":"2024-08-17T02:45:20.031922+0000","last_fullsized":"2024-08-17T02:45:20.031922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:37:57.900723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.013832+0000","last_change":"2024-08-17T02:45:19.046461+0000","last_active":"2024-08-17T02:45:20.013832+0000","last_peered":"2024-08-17T02:45:20.013832+0000","last_clean":"2024-08-17T02:45:20.013832+0000","last_became_active":"2024-08-17T02:45:19.042611+0000","last_became_peered":"2024-08-17T02:45:19.042611+0000","last_unstale":"2024-08-17T02:45:20.013832+0000","last_undegraded":"2024-08-17T02:45:20.013832+0000","last_fullsized":"2024-08-17T02:45:20.013832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:41:32.772441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052264+0000","last_change":"2024-08-17T02:45:20.043147+0000","last_active":"2024-08-17T02:45:20.052264+0000","last_peered":"2024-08-17T02:45:20.052264+0000","last_clean":"2024-08-17T02:45:20.052264+0000","last_became_active":"2024-08-17T02:45:20.042828+0000","last_became_peered":"2024-08-17T02:45:20.042828+0000","last_unstale":"2024-08-17T02:45:20.052264+0000","last_undegraded":"2024-08-17T02:45:20.052264+0000","last_fullsized":"2024-08-17T02:45:20.052264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:30.359495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.031941+0000","last_change":"2024-08-17T02:45:19.047594+0000","last_active":"2024-08-17T02:45:20.031941+0000","last_peered":"2024-08-17T02:45:20.031941+0000","last_clean":"2024-08-17T02:45:20.031941+0000","last_became_active":"2024-08-17T02:45:19.047271+0000","last_became_peered":"2024-08-17T02:45:19.047271+0000","last_unstale":"2024-08-17T02:45:20.031941+0000","last_undegraded":"2024-08-17T02:45:20.031941+0000","last_fullsized":"2024-08-17T02:45:20.031941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.050654+0000","last_change":"2024-08-17T02:45:20.043304+0000","last_active":"2024-08-17T02:45:20.050654+0000","last_peered":"2024-08-17T02:45:20.050654+0000","last_clean":"2024-08-17T02:45:20.050654+0000","last_became_active":"2024-08-17T02:45:20.038490+0000","last_became_peered":"2024-08-17T02:45:20.038490+0000","last_unstale":"2024-08-17T02:45:20.050654+0000","last_undegraded":"2024-08-17T02:45:20.050654+0000","last_fullsized":"2024-08-17T02:45:20.050654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:00:06.622218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.014144+0000","last_change":"2024-08-17T02:45:19.048638+0000","last_active":"2024-08-17T02:45:20.014144+0000","last_peered":"2024-08-17T02:45:20.014144+0000","last_clean":"2024-08-17T02:45:20.014144+0000","last_became_active":"2024-08-17T02:45:19.048346+0000","last_became_peered":"2024-08-17T02:45:19.048346+0000","last_unstale":"2024-08-17T02:45:20.014144+0000","last_undegraded":"2024-08-17T02:45:20.014144+0000","last_fullsized":"2024-08-17T02:45:20.014144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:46:09.850664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.034371+0000","last_change":"2024-08-17T02:45:20.030375+0000","last_active":"2024-08-17T02:45:20.034371+0000","last_peered":"2024-08-17T02:45:20.034371+0000","last_clean":"2024-08-17T02:45:20.034371+0000","last_became_active":"2024-08-17T02:45:20.029966+0000","last_became_peered":"2024-08-17T02:45:20.029966+0000","last_unstale":"2024-08-17T02:45:20.034371+0000","last_undegraded":"2024-08-17T02:45:20.034371+0000","last_fullsized":"2024-08-17T02:45:20.034371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:16:04.128578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.021330+0000","last_change":"2024-08-17T02:45:19.039495+0000","last_active":"2024-08-17T02:45:20.021330+0000","last_peered":"2024-08-17T02:45:20.021330+0000","last_clean":"2024-08-17T02:45:20.021330+0000","last_became_active":"2024-08-17T02:45:19.039218+0000","last_became_peered":"2024-08-17T02:45:19.039218+0000","last_unstale":"2024-08-17T02:45:20.021330+0000","last_undegraded":"2024-08-17T02:45:20.021330+0000","last_fullsized":"2024-08-17T02:45:20.021330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052935+0000","last_change":"2024-08-17T02:45:20.047183+0000","last_active":"2024-08-17T02:45:20.052935+0000","last_peered":"2024-08-17T02:45:20.052935+0000","last_clean":"2024-08-17T02:45:20.052935+0000","last_became_active":"2024-08-17T02:45:20.045627+0000","last_became_peered":"2024-08-17T02:45:20.045627+0000","last_unstale":"2024-08-17T02:45:20.052935+0000","last_undegraded":"2024-08-17T02:45:20.052935+0000","last_fullsized":"2024-08-17T02:45:20.052935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:02:17.358927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.016227+0000","last_change":"2024-08-17T02:45:19.043843+0000","last_active":"2024-08-17T02:45:20.016227+0000","last_peered":"2024-08-17T02:45:20.016227+0000","last_clean":"2024-08-17T02:45:20.016227+0000","last_became_active":"2024-08-17T02:45:19.043069+0000","last_became_peered":"2024-08-17T02:45:19.043069+0000","last_unstale":"2024-08-17T02:45:20.016227+0000","last_undegraded":"2024-08-17T02:45:20.016227+0000","last_fullsized":"2024-08-17T02:45:20.016227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:43.862231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051578+0000","last_change":"2024-08-17T02:45:20.038816+0000","last_active":"2024-08-17T02:45:20.051578+0000","last_peered":"2024-08-17T02:45:20.051578+0000","last_clean":"2024-08-17T02:45:20.051578+0000","last_became_active":"2024-08-17T02:45:20.038520+0000","last_became_peered":"2024-08-17T02:45:20.038520+0000","last_unstale":"2024-08-17T02:45:20.051578+0000","last_undegraded":"2024-08-17T02:45:20.051578+0000","last_fullsized":"2024-08-17T02:45:20.051578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:37:57.900723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.016603+0000","last_change":"2024-08-17T02:45:19.044921+0000","last_active":"2024-08-17T02:45:20.016603+0000","last_peered":"2024-08-17T02:45:20.016603+0000","last_clean":"2024-08-17T02:45:20.016603+0000","last_became_active":"2024-08-17T02:45:19.041305+0000","last_became_peered":"2024-08-17T02:45:19.041305+0000","last_unstale":"2024-08-17T02:45:20.016603+0000","last_undegraded":"2024-08-17T02:45:20.016603+0000","last_fullsized":"2024-08-17T02:45:20.016603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:03:27.020817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.041518+0000","last_change":"2024-08-17T02:45:20.035169+0000","last_active":"2024-08-17T02:45:20.041518+0000","last_peered":"2024-08-17T02:45:20.041518+0000","last_clean":"2024-08-17T02:45:20.041518+0000","last_became_active":"2024-08-17T02:45:20.034858+0000","last_became_peered":"2024-08-17T02:45:20.034858+0000","last_unstale":"2024-08-17T02:45:20.041518+0000","last_undegraded":"2024-08-17T02:45:20.041518+0000","last_fullsized":"2024-08-17T02:45:20.041518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:53:40.660806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054541+0000","last_change":"2024-08-17T02:45:19.049505+0000","last_active":"2024-08-17T02:45:20.054541+0000","last_peered":"2024-08-17T02:45:20.054541+0000","last_clean":"2024-08-17T02:45:20.054541+0000","last_became_active":"2024-08-17T02:45:19.048843+0000","last_became_peered":"2024-08-17T02:45:19.048843+0000","last_unstale":"2024-08-17T02:45:20.054541+0000","last_undegraded":"2024-08-17T02:45:20.054541+0000","last_fullsized":"2024-08-17T02:45:20.054541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:53:39.658967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.046988+0000","last_change":"2024-08-17T02:45:20.038861+0000","last_active":"2024-08-17T02:45:20.046988+0000","last_peered":"2024-08-17T02:45:20.046988+0000","last_clean":"2024-08-17T02:45:20.046988+0000","last_became_active":"2024-08-17T02:45:20.038597+0000","last_became_peered":"2024-08-17T02:45:20.038597+0000","last_unstale":"2024-08-17T02:45:20.046988+0000","last_undegraded":"2024-08-17T02:45:20.046988+0000","last_fullsized":"2024-08-17T02:45:20.046988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:02:55.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.034402+0000","last_change":"2024-08-17T02:45:19.035699+0000","last_active":"2024-08-17T02:45:20.034402+0000","last_peered":"2024-08-17T02:45:20.034402+0000","last_clean":"2024-08-17T02:45:20.034402+0000","last_became_active":"2024-08-17T02:45:19.035429+0000","last_became_peered":"2024-08-17T02:45:19.035429+0000","last_unstale":"2024-08-17T02:45:20.034402+0000","last_undegraded":"2024-08-17T02:45:20.034402+0000","last_fullsized":"2024-08-17T02:45:20.034402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:41:32.772441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.021682+0000","last_change":"2024-08-17T02:45:20.020728+0000","last_active":"2024-08-17T02:45:20.021682+0000","last_peered":"2024-08-17T02:45:20.021682+0000","last_clean":"2024-08-17T02:45:20.021682+0000","last_became_active":"2024-08-17T02:45:20.020439+0000","last_became_peered":"2024-08-17T02:45:20.020439+0000","last_unstale":"2024-08-17T02:45:20.021682+0000","last_undegraded":"2024-08-17T02:45:20.021682+0000","last_fullsized":"2024-08-17T02:45:20.021682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:32:46.909589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.016454+0000","last_change":"2024-08-17T02:45:19.044294+0000","last_active":"2024-08-17T02:45:20.016454+0000","last_peered":"2024-08-17T02:45:20.016454+0000","last_clean":"2024-08-17T02:45:20.016454+0000","last_became_active":"2024-08-17T02:45:19.044001+0000","last_became_peered":"2024-08-17T02:45:19.044001+0000","last_unstale":"2024-08-17T02:45:20.016454+0000","last_undegraded":"2024-08-17T02:45:20.016454+0000","last_fullsized":"2024-08-17T02:45:20.016454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:43.862231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047889+0000","last_change":"2024-08-17T02:45:20.040445+0000","last_active":"2024-08-17T02:45:20.047889+0000","last_peered":"2024-08-17T02:45:20.047889+0000","last_clean":"2024-08-17T02:45:20.047889+0000","last_became_active":"2024-08-17T02:45:20.040151+0000","last_became_peered":"2024-08-17T02:45:20.040151+0000","last_unstale":"2024-08-17T02:45:20.047889+0000","last_undegraded":"2024-08-17T02:45:20.047889+0000","last_fullsized":"2024-08-17T02:45:20.047889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:55:18.896035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.033393+0000","last_change":"2024-08-17T02:45:19.037364+0000","last_active":"2024-08-17T02:45:20.033393+0000","last_peered":"2024-08-17T02:45:20.033393+0000","last_clean":"2024-08-17T02:45:20.033393+0000","last_became_active":"2024-08-17T02:45:19.037020+0000","last_became_peered":"2024-08-17T02:45:19.037020+0000","last_unstale":"2024-08-17T02:45:20.033393+0000","last_undegraded":"2024-08-17T02:45:20.033393+0000","last_fullsized":"2024-08-17T02:45:20.033393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052107+0000","last_change":"2024-08-17T02:45:20.047671+0000","last_active":"2024-08-17T02:45:20.052107+0000","last_peered":"2024-08-17T02:45:20.052107+0000","last_clean":"2024-08-17T02:45:20.052107+0000","last_became_active":"2024-08-17T02:45:20.046596+0000","last_became_peered":"2024-08-17T02:45:20.046596+0000","last_unstale":"2024-08-17T02:45:20.052107+0000","last_undegraded":"2024-08-17T02:45:20.052107+0000","last_fullsized":"2024-08-17T02:45:20.052107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:46:10.852503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.057610+0000","last_change":"2024-08-17T02:45:19.040794+0000","last_active":"2024-08-17T02:45:20.057610+0000","last_peered":"2024-08-17T02:45:20.057610+0000","last_clean":"2024-08-17T02:45:20.057610+0000","last_became_active":"2024-08-17T02:45:19.038764+0000","last_became_peered":"2024-08-17T02:45:19.038764+0000","last_unstale":"2024-08-17T02:45:20.057610+0000","last_undegraded":"2024-08-17T02:45:20.057610+0000","last_fullsized":"2024-08-17T02:45:20.057610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:04:43.204736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.021022+0000","last_change":"2024-08-17T02:45:20.019301+0000","last_active":"2024-08-17T02:45:20.021022+0000","last_peered":"2024-08-17T02:45:20.021022+0000","last_clean":"2024-08-17T02:45:20.021022+0000","last_became_active":"2024-08-17T02:45:20.018641+0000","last_became_peered":"2024-08-17T02:45:20.018641+0000","last_unstale":"2024-08-17T02:45:20.021022+0000","last_undegraded":"2024-08-17T02:45:20.021022+0000","last_fullsized":"2024-08-17T02:45:20.021022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:47:06.908123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052207+0000","last_change":"2024-08-17T02:45:20.037807+0000","last_active":"2024-08-17T02:45:20.052207+0000","last_peered":"2024-08-17T02:45:20.052207+0000","last_clean":"2024-08-17T02:45:20.052207+0000","last_became_active":"2024-08-17T02:45:20.034485+0000","last_became_peered":"2024-08-17T02:45:20.034485+0000","last_unstale":"2024-08-17T02:45:20.052207+0000","last_undegraded":"2024-08-17T02:45:20.052207+0000","last_fullsized":"2024-08-17T02:45:20.052207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:29:54.794188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.055868+0000","last_change":"2024-08-17T02:45:19.046881+0000","last_active":"2024-08-17T02:45:20.055868+0000","last_peered":"2024-08-17T02:45:20.055868+0000","last_clean":"2024-08-17T02:45:20.055868+0000","last_became_active":"2024-08-17T02:45:19.046676+0000","last_became_peered":"2024-08-17T02:45:19.046676+0000","last_unstale":"2024-08-17T02:45:20.055868+0000","last_undegraded":"2024-08-17T02:45:20.055868+0000","last_fullsized":"2024-08-17T02:45:20.055868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.041726+0000","last_change":"2024-08-17T02:45:20.040519+0000","last_active":"2024-08-17T02:45:20.041726+0000","last_peered":"2024-08-17T02:45:20.041726+0000","last_clean":"2024-08-17T02:45:20.041726+0000","last_became_active":"2024-08-17T02:45:20.040246+0000","last_became_peered":"2024-08-17T02:45:20.040246+0000","last_unstale":"2024-08-17T02:45:20.041726+0000","last_undegraded":"2024-08-17T02:45:20.041726+0000","last_fullsized":"2024-08-17T02:45:20.041726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:46:10.852503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.055789+0000","last_change":"2024-08-17T02:45:19.045919+0000","last_active":"2024-08-17T02:45:20.055789+0000","last_peered":"2024-08-17T02:45:20.055789+0000","last_clean":"2024-08-17T02:45:20.055789+0000","last_became_active":"2024-08-17T02:45:19.045693+0000","last_became_peered":"2024-08-17T02:45:19.045693+0000","last_unstale":"2024-08-17T02:45:20.055789+0000","last_undegraded":"2024-08-17T02:45:20.055789+0000","last_fullsized":"2024-08-17T02:45:20.055789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:55:24.029624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.050681+0000","last_change":"2024-08-17T02:45:20.044015+0000","last_active":"2024-08-17T02:45:20.050681+0000","last_peered":"2024-08-17T02:45:20.050681+0000","last_clean":"2024-08-17T02:45:20.050681+0000","last_became_active":"2024-08-17T02:45:20.043471+0000","last_became_peered":"2024-08-17T02:45:20.043471+0000","last_unstale":"2024-08-17T02:45:20.050681+0000","last_undegraded":"2024-08-17T02:45:20.050681+0000","last_fullsized":"2024-08-17T02:45:20.050681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:20.121506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054182+0000","last_change":"2024-08-17T02:45:19.041823+0000","last_active":"2024-08-17T02:45:20.054182+0000","last_peered":"2024-08-17T02:45:20.054182+0000","last_clean":"2024-08-17T02:45:20.054182+0000","last_became_active":"2024-08-17T02:45:19.041481+0000","last_became_peered":"2024-08-17T02:45:19.041481+0000","last_unstale":"2024-08-17T02:45:20.054182+0000","last_undegraded":"2024-08-17T02:45:20.054182+0000","last_fullsized":"2024-08-17T02:45:20.054182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:03:27.020817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051425+0000","last_change":"2024-08-17T02:45:20.047339+0000","last_active":"2024-08-17T02:45:20.051425+0000","last_peered":"2024-08-17T02:45:20.051425+0000","last_clean":"2024-08-17T02:45:20.051425+0000","last_became_active":"2024-08-17T02:45:20.047111+0000","last_became_peered":"2024-08-17T02:45:20.047111+0000","last_unstale":"2024-08-17T02:45:20.051425+0000","last_undegraded":"2024-08-17T02:45:20.051425+0000","last_fullsized":"2024-08-17T02:45:20.051425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:46:59.247481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054521+0000","last_change":"2024-08-17T02:45:19.042331+0000","last_active":"2024-08-17T02:45:20.054521+0000","last_peered":"2024-08-17T02:45:20.054521+0000","last_clean":"2024-08-17T02:45:20.054521+0000","last_became_active":"2024-08-17T02:45:19.042033+0000","last_became_peered":"2024-08-17T02:45:19.042033+0000","last_unstale":"2024-08-17T02:45:20.054521+0000","last_undegraded":"2024-08-17T02:45:20.054521+0000","last_fullsized":"2024-08-17T02:45:20.054521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:25:12.592566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":75437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":217088,"data_stored":75313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298156,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073684,"statfs":{"total":96636764160,"available":96331452416,"internally_reserved":0,"allocated":819200,"data_stored":665773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":237568,"data_stored":77525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":221184,"data_stored":75883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301744,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070096,"statfs":{"total":96636764160,"available":96327778304,"internally_reserved":0,"allocated":229376,"data_stored":76865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":815104,"data_stored":665751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301740,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070100,"statfs":{"total":96636764160,"available":96327782400,"internally_reserved":0,"allocated":225280,"data_stored":75369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:45:24.636 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-08-17T02:45:24.637 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-17T02:45:24.637 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-08-17T02:45:24.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-08-17T02:45:24.637 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-08-17T02:45:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.822+0000 7f2037e00640 1 Processor -- start 2024-08-17T02:45:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.822+0000 7f2037e00640 1 -- start start 2024-08-17T02:45:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.823+0000 7f2037e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20380aed50 0x7f20380ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.823+0000 7f2037e00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20380cc890 con 0x7f20380c5c40 2024-08-17T02:45:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.823+0000 7f2037e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20380ceb20 con 0x7f20380ce6d0 2024-08-17T02:45:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.823+0000 7f2037e00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20380cb4c0 con 0x7f20380aed50 2024-08-17T02:45:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.823+0000 7f2036a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20380aed50 0x7f20380ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.823+0000 7f2037400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f20380c5c40 0x7f20381627a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46100/0 (socket says 172.21.15.16:46100) 2024-08-17T02:45:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.823+0000 7f2037400640 1 -- 172.21.15.16:0/101874674 learned_addr learned my addr 172.21.15.16:0/101874674 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.824+0000 7f2036a00640 1 -- 172.21.15.16:0/101874674 >> v1:172.21.15.142:6789/0 conn(0x7f20380ce6d0 legacy=0x7f20380ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.824+0000 7f2036a00640 1 -- 172.21.15.16:0/101874674 >> v1:172.21.15.16:6789/0 conn(0x7f20380c5c40 legacy=0x7f20381627a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.824+0000 7f2036a00640 1 -- 172.21.15.16:0/101874674 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20380cb7a0 con 0x7f20380aed50 2024-08-17T02:45:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.824+0000 7f2036000640 1 -- 172.21.15.16:0/101874674 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055268956 0 0) 0x7f20380cc890 con 0x7f20380c5c40 2024-08-17T02:45:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.825+0000 7f2036a00640 1 --2- 172.21.15.16:0/101874674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380aed50 0x7f20380ce190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2028009850 tx=0x7f20280090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59d356a51c965b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.825+0000 7f2036000640 1 -- 172.21.15.16:0/101874674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202802adb0 con 0x7f20380aed50 2024-08-17T02:45:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.825+0000 7f2036000640 1 -- 172.21.15.16:0/101874674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2028032440 con 0x7f20380aed50 2024-08-17T02:45:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.825+0000 7f2036000640 1 -- 172.21.15.16:0/101874674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20280327e0 con 0x7f20380aed50 2024-08-17T02:45:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.825+0000 7f2037e00640 1 -- 172.21.15.16:0/101874674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380aed50 msgr2=0x7f20380ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.826+0000 7f2037e00640 1 --2- 172.21.15.16:0/101874674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380aed50 0x7f20380ce190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2028009850 tx=0x7f20280090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.826+0000 7f2037e00640 1 -- 172.21.15.16:0/101874674 shutdown_connections 2024-08-17T02:45:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.826+0000 7f2037e00640 1 --2- 172.21.15.16:0/101874674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380aed50 0x7f20380ce190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.826+0000 7f2037e00640 1 -- 172.21.15.16:0/101874674 >> 172.21.15.16:0/101874674 conn(0x7f2038098e30 msgr2=0x7f203806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.826+0000 7f2037e00640 1 -- 172.21.15.16:0/101874674 shutdown_connections 2024-08-17T02:45:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.826+0000 7f2037e00640 1 -- 172.21.15.16:0/101874674 wait complete. 2024-08-17T02:45:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.827+0000 7f2037e00640 1 Processor -- start 2024-08-17T02:45:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.827+0000 7f2037e00640 1 -- start start 2024-08-17T02:45:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.828+0000 7f2037e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20380aed50 0x7f20380b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.828+0000 7f2037e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20380c5c40 0x7f203814a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.828+0000 7f2037e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380ce6d0 0x7f20380b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.828+0000 7f2037e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20380c9a20 con 0x7f20380ce6d0 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.828+0000 7f2037e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20380c7100 con 0x7f20380aed50 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.828+0000 7f2037e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20380c9460 con 0x7f20380c5c40 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.828+0000 7f2037400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380ce6d0 0x7f20380b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.828+0000 7f2037400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380ce6d0 0x7f20380b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57848/0 (socket says 172.21.15.16:57848) 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.828+0000 7f2037400640 1 -- 172.21.15.16:0/647623021 learned_addr learned my addr 172.21.15.16:0/647623021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f2036a00640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20380aed50 0x7f20380b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f202fe00640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20380c5c40 0x7f203814a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f2037400640 1 -- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20380c5c40 msgr2=0x7f203814a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f2037400640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20380c5c40 0x7f203814a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f2037400640 1 -- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20380aed50 msgr2=0x7f20380b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f2037400640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20380aed50 0x7f20380b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f2037400640 1 -- 172.21.15.16:0/647623021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2028009390 con 0x7f20380ce6d0 2024-08-17T02:45:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f2036a00640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20380aed50 0x7f20380b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f2037400640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380ce6d0 0x7f20380b4630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f203000cc10 tx=0x7f2030001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f202e000640 1 -- 172.21.15.16:0/647623021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203000ce30 con 0x7f20380ce6d0 2024-08-17T02:45:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f202e000640 1 -- 172.21.15.16:0/647623021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203000a7f0 con 0x7f20380ce6d0 2024-08-17T02:45:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f202e000640 1 -- 172.21.15.16:0/647623021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203000abf0 con 0x7f20380ce6d0 2024-08-17T02:45:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f2037e00640 1 -- 172.21.15.16:0/647623021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20380c9740 con 0x7f20380ce6d0 2024-08-17T02:45:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.829+0000 7f2037e00640 1 -- 172.21.15.16:0/647623021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20380b3c10 con 0x7f20380ce6d0 2024-08-17T02:45:24.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.831+0000 7f2037e00640 1 -- 172.21.15.16:0/647623021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20380c7270 con 0x7f20380ce6d0 2024-08-17T02:45:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.831+0000 7f202e000640 1 -- 172.21.15.16:0/647623021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f203000ad50 con 0x7f20380ce6d0 2024-08-17T02:45:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.832+0000 7f202e000640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2014065150 0x7f2014067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.832+0000 7f202e000640 1 -- 172.21.15.16:0/647623021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2030082580 con 0x7f20380ce6d0 2024-08-17T02:45:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.833+0000 7f2036a00640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2014065150 0x7f2014067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:24.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.833+0000 7f2036a00640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2014065150 0x7f2014067600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2028034000 tx=0x7f2028032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:24.836+0000 7f202e000640 1 -- 172.21.15.16:0/647623021 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f20300506e0 con 0x7f20380ce6d0 2024-08-17T02:45:25.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.067+0000 7f2037e00640 1 -- 172.21.15.16:0/647623021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f2038071050 con 0x7f20380ce6d0 2024-08-17T02:45:25.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.070+0000 7f202e000640 1 -- 172.21.15.16:0/647623021 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+161036 (secure 0 0 0) 0x7f2030053d00 con 0x7f20380ce6d0 2024-08-17T02:45:25.074 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:25.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.073+0000 7f2037e00640 1 -- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2014065150 msgr2=0x7f2014067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.073+0000 7f2037e00640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2014065150 0x7f2014067600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2028034000 tx=0x7f2028032000 comp rx=0 tx=0).stop 2024-08-17T02:45:25.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.073+0000 7f2037e00640 1 -- 172.21.15.16:0/647623021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380ce6d0 msgr2=0x7f20380b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.073+0000 7f2037e00640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380ce6d0 0x7f20380b4630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f203000cc10 tx=0x7f2030001970 comp rx=0 tx=0).stop 2024-08-17T02:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.073+0000 7f2037e00640 1 -- 172.21.15.16:0/647623021 shutdown_connections 2024-08-17T02:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.073+0000 7f2037e00640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2014065150 0x7f2014067600 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.073+0000 7f2037e00640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20380ce6d0 0x7f20380b4630 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.073+0000 7f2037e00640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f20380c5c40 0x7f203814a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.073+0000 7f2037e00640 1 --2- 172.21.15.16:0/647623021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20380aed50 0x7f20380b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.073+0000 7f2037e00640 1 -- 172.21.15.16:0/647623021 >> 172.21.15.16:0/647623021 conn(0x7f2038098e30 msgr2=0x7f2038158270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.073+0000 7f2037e00640 1 -- 172.21.15.16:0/647623021 shutdown_connections 2024-08-17T02:45:25.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.073+0000 7f2037e00640 1 -- 172.21.15.16:0/647623021 wait complete. 2024-08-17T02:45:25.089 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":37028},{"type":"v1","addr":"172.21.15.142:6833","nonce":37028}]},"active_addr":"172.21.15.142:6833/37028","active_change":"2024-08-17T02:45:06.222694+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":516414127}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":474103209}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":1212085822}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.142:0","nonce":1477591154}]}]} 2024-08-17T02:45:25.093 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-08-17T02:45:25.093 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-08-17T02:45:25.093 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.241+0000 7f21b5800640 1 Processor -- start 2024-08-17T02:45:25.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.241+0000 7f21b5800640 1 -- start start 2024-08-17T02:45:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.242+0000 7f21b5800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f21b01401b0 0x7f21b00caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.243+0000 7f21b5800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21b00b1440 0x7f21b00c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.243+0000 7f21b5800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b00caa80 con 0x7f21b0140580 2024-08-17T02:45:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.243+0000 7f21b5800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b00cb450 con 0x7f21b00b1440 2024-08-17T02:45:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.243+0000 7f21b5800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b00c9e70 con 0x7f21b01401b0 2024-08-17T02:45:25.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.243+0000 7f21af400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21b00b1440 0x7f21b00c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:25.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.243+0000 7f21a7e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f21b0140580 0x7f21b00b0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46114/0 (socket says 172.21.15.16:46114) 2024-08-17T02:45:25.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.243+0000 7f21a7e00640 1 -- 172.21.15.16:0/1372880775 learned_addr learned my addr 172.21.15.16:0/1372880775 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:25.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.243+0000 7f21aea00640 1 --2- 172.21.15.16:0/1372880775 >> v2:172.21.15.142:3300/0 conn(0x7f21b01401b0 0x7f21b00caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:25.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.243+0000 7f21af400640 1 -- 172.21.15.16:0/1372880775 >> v2:172.21.15.142:3300/0 conn(0x7f21b01401b0 msgr2=0x7f21b00caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:25.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.243+0000 7f21af400640 1 --2- 172.21.15.16:0/1372880775 >> v2:172.21.15.142:3300/0 conn(0x7f21b01401b0 0x7f21b00caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.243+0000 7f21af400640 1 -- 172.21.15.16:0/1372880775 >> v1:172.21.15.16:6789/0 conn(0x7f21b0140580 legacy=0x7f21b00b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.243+0000 7f21af400640 1 -- 172.21.15.16:0/1372880775 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b00ca150 con 0x7f21b00b1440 2024-08-17T02:45:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.244+0000 7f21af400640 1 --2- 172.21.15.16:0/1372880775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b00b1440 0x7f21b00c9040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f21b0068650 tx=0x7f21a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca41e9c631ca66c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:25.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.244+0000 7f21ae000640 1 -- 172.21.15.16:0/1372880775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8007550 con 0x7f21b00b1440 2024-08-17T02:45:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.245+0000 7f21ae000640 1 -- 172.21.15.16:0/1372880775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a802d050 con 0x7f21b00b1440 2024-08-17T02:45:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.245+0000 7f21ae000640 1 -- 172.21.15.16:0/1372880775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8007920 con 0x7f21b00b1440 2024-08-17T02:45:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.245+0000 7f21b5800640 1 -- 172.21.15.16:0/1372880775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b00b1440 msgr2=0x7f21b00c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.245+0000 7f21b5800640 1 --2- 172.21.15.16:0/1372880775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b00b1440 0x7f21b00c9040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f21b0068650 tx=0x7f21a8009780 comp rx=0 tx=0).stop 2024-08-17T02:45:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.245+0000 7f21b5800640 1 -- 172.21.15.16:0/1372880775 shutdown_connections 2024-08-17T02:45:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.245+0000 7f21b5800640 1 --2- 172.21.15.16:0/1372880775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b00b1440 0x7f21b00c9040 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.245+0000 7f21b5800640 1 --2- 172.21.15.16:0/1372880775 >> v2:172.21.15.142:3300/0 conn(0x7f21b01401b0 0x7f21b00caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.245+0000 7f21b5800640 1 -- 172.21.15.16:0/1372880775 >> 172.21.15.16:0/1372880775 conn(0x7f21b00cb9b0 msgr2=0x7f21b00cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.245+0000 7f21b5800640 1 -- 172.21.15.16:0/1372880775 shutdown_connections 2024-08-17T02:45:25.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.245+0000 7f21b5800640 1 -- 172.21.15.16:0/1372880775 wait complete. 2024-08-17T02:45:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.247+0000 7f21b5800640 1 Processor -- start 2024-08-17T02:45:25.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.247+0000 7f21b5800640 1 -- start start 2024-08-17T02:45:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21b5800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21b00b1440 0x7f21b016bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:25.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21b5800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b01401b0 0x7f21b00c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21b5800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21b0140580 0x7f21b00c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21b5800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b00c5ca0 con 0x7f21b01401b0 2024-08-17T02:45:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21b5800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b00c7730 con 0x7f21b0140580 2024-08-17T02:45:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21b5800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b013b450 con 0x7f21b00b1440 2024-08-17T02:45:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b01401b0 0x7f21b00c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21a7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b01401b0 0x7f21b00c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57856/0 (socket says 172.21.15.16:57856) 2024-08-17T02:45:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21a7e00640 1 -- 172.21.15.16:0/2565654898 learned_addr learned my addr 172.21.15.16:0/2565654898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21aea00640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21b00b1440 0x7f21b016bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21af400640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21b0140580 0x7f21b00c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21a7e00640 1 -- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21b00b1440 msgr2=0x7f21b016bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21a7e00640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21b00b1440 0x7f21b016bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21a7e00640 1 -- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21b0140580 msgr2=0x7f21b00c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21a7e00640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21b0140580 0x7f21b00c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.248+0000 7f21a7e00640 1 -- 172.21.15.16:0/2565654898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a8009a20 con 0x7f21b01401b0 2024-08-17T02:45:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.249+0000 7f21a7e00640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b01401b0 0x7f21b00c5760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f219800c5c0 tx=0x7f2198007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.249+0000 7f21a6000640 1 -- 172.21.15.16:0/2565654898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198005aa0 con 0x7f21b01401b0 2024-08-17T02:45:25.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.249+0000 7f21a6000640 1 -- 172.21.15.16:0/2565654898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2198005c20 con 0x7f21b01401b0 2024-08-17T02:45:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.249+0000 7f21a6000640 1 -- 172.21.15.16:0/2565654898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21980070f0 con 0x7f21b01401b0 2024-08-17T02:45:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.249+0000 7f21b5800640 1 -- 172.21.15.16:0/2565654898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b00c5ec0 con 0x7f21b01401b0 2024-08-17T02:45:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.249+0000 7f21b5800640 1 -- 172.21.15.16:0/2565654898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b013ae00 con 0x7f21b01401b0 2024-08-17T02:45:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.250+0000 7f21b5800640 1 -- 172.21.15.16:0/2565654898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b013b5c0 con 0x7f21b01401b0 2024-08-17T02:45:25.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.251+0000 7f21a6000640 1 -- 172.21.15.16:0/2565654898 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f2198007520 con 0x7f21b01401b0 2024-08-17T02:45:25.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.252+0000 7f21a6000640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f218c065150 0x7f218c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:25.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.252+0000 7f21a6000640 1 -- 172.21.15.16:0/2565654898 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2198081460 con 0x7f21b01401b0 2024-08-17T02:45:25.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.252+0000 7f21aea00640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f218c065150 0x7f218c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:25.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.253+0000 7f21aea00640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f218c065150 0x7f218c067600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f21b013d6a0 tx=0x7f21a0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:25.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.256+0000 7f21a6000640 1 -- 172.21.15.16:0/2565654898 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f219804f650 con 0x7f21b01401b0 2024-08-17T02:45:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.440+0000 7f21b5800640 1 -- 172.21.15.16:0/2565654898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21b0064db0 con 0x7f21b01401b0 2024-08-17T02:45:25.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.441+0000 7f21a6000640 1 -- 172.21.15.16:0/2565654898 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f2198052c70 con 0x7f21b01401b0 2024-08-17T02:45:25.443 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:25.443 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.443+0000 7f21b5800640 1 -- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f218c065150 msgr2=0x7f218c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.443+0000 7f21b5800640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f218c065150 0x7f218c067600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f21b013d6a0 tx=0x7f21a0008040 comp rx=0 tx=0).stop 2024-08-17T02:45:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.443+0000 7f21b5800640 1 -- 172.21.15.16:0/2565654898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b01401b0 msgr2=0x7f21b00c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.443+0000 7f21b5800640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b01401b0 0x7f21b00c5760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f219800c5c0 tx=0x7f2198007ee0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.443+0000 7f21b5800640 1 -- 172.21.15.16:0/2565654898 shutdown_connections 2024-08-17T02:45:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.443+0000 7f21b5800640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f218c065150 0x7f218c067600 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.443+0000 7f21b5800640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21b0140580 0x7f21b00c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.443+0000 7f21b5800640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21b01401b0 0x7f21b00c5760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.443+0000 7f21b5800640 1 --2- 172.21.15.16:0/2565654898 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21b00b1440 0x7f21b016bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.443+0000 7f21b5800640 1 -- 172.21.15.16:0/2565654898 >> 172.21.15.16:0/2565654898 conn(0x7f21b00cb9b0 msgr2=0x7f21b00aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.443+0000 7f21b5800640 1 -- 172.21.15.16:0/2565654898 shutdown_connections 2024-08-17T02:45:25.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.444+0000 7f21b5800640 1 -- 172.21.15.16:0/2565654898 wait complete. 2024-08-17T02:45:25.460 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-08-17T02:45:25.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:25.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.610+0000 7f57a7400640 1 Processor -- start 2024-08-17T02:45:25.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.610+0000 7f57a7400640 1 -- start start 2024-08-17T02:45:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.611+0000 7f57a7400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f57a0155a20 0x7f57a00adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.612+0000 7f57a7400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57a006df00 0x7f57a00ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.612+0000 7f57a7400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a0156670 con 0x7f57a0155df0 2024-08-17T02:45:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.612+0000 7f57a7400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a00b0350 con 0x7f57a006df00 2024-08-17T02:45:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.612+0000 7f57a7400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a006e5e0 con 0x7f57a0155a20 2024-08-17T02:45:25.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.612+0000 7f57a5400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57a006df00 0x7f57a00ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.612+0000 7f57a5400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57a006df00 0x7f57a00ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57860/0 (socket says 172.21.15.16:57860) 2024-08-17T02:45:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.612+0000 7f57a5400640 1 -- 172.21.15.16:0/3912845879 learned_addr learned my addr 172.21.15.16:0/3912845879 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.612+0000 7f57a4a00640 1 --2- 172.21.15.16:0/3912845879 >> v2:172.21.15.142:3300/0 conn(0x7f57a0155a20 0x7f57a00adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.612+0000 7f57a5400640 1 -- 172.21.15.16:0/3912845879 >> v2:172.21.15.142:3300/0 conn(0x7f57a0155a20 msgr2=0x7f57a00adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.612+0000 7f57a5400640 1 --2- 172.21.15.16:0/3912845879 >> v2:172.21.15.142:3300/0 conn(0x7f57a0155a20 0x7f57a00adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.612+0000 7f57a5400640 1 -- 172.21.15.16:0/3912845879 >> v1:172.21.15.142:6789/0 conn(0x7f57a0155df0 legacy=0x7f57a006d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:25.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.612+0000 7f57a5400640 1 -- 172.21.15.16:0/3912845879 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a006e8c0 con 0x7f57a006df00 2024-08-17T02:45:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.613+0000 7f57a5400640 1 --2- 172.21.15.16:0/3912845879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a006df00 0x7f57a00ad9b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f57a006bc00 tx=0x7f5798009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498c82192931b844 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.613+0000 7f579fe00640 1 -- 172.21.15.16:0/3912845879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798031400 con 0x7f57a006df00 2024-08-17T02:45:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.614+0000 7f579fe00640 1 -- 172.21.15.16:0/3912845879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579802d050 con 0x7f57a006df00 2024-08-17T02:45:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.614+0000 7f579fe00640 1 -- 172.21.15.16:0/3912845879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980317d0 con 0x7f57a006df00 2024-08-17T02:45:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.614+0000 7f57a7400640 1 -- 172.21.15.16:0/3912845879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a006df00 msgr2=0x7f57a00ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.614+0000 7f57a7400640 1 --2- 172.21.15.16:0/3912845879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a006df00 0x7f57a00ad9b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f57a006bc00 tx=0x7f5798009840 comp rx=0 tx=0).stop 2024-08-17T02:45:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.614+0000 7f57a7400640 1 -- 172.21.15.16:0/3912845879 shutdown_connections 2024-08-17T02:45:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.614+0000 7f57a7400640 1 --2- 172.21.15.16:0/3912845879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a006df00 0x7f57a00ad9b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.614+0000 7f57a7400640 1 --2- 172.21.15.16:0/3912845879 >> v2:172.21.15.142:3300/0 conn(0x7f57a0155a20 0x7f57a00adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.614+0000 7f57a7400640 1 -- 172.21.15.16:0/3912845879 >> 172.21.15.16:0/3912845879 conn(0x7f57a00cb9b0 msgr2=0x7f57a00cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.614+0000 7f57a7400640 1 -- 172.21.15.16:0/3912845879 shutdown_connections 2024-08-17T02:45:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.614+0000 7f57a7400640 1 -- 172.21.15.16:0/3912845879 wait complete. 2024-08-17T02:45:25.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.616+0000 7f57a7400640 1 Processor -- start 2024-08-17T02:45:25.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.616+0000 7f57a7400640 1 -- start start 2024-08-17T02:45:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.616+0000 7f57a7400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a006df00 0x7f57a014d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.616+0000 7f57a7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a0155a20 0x7f57a015f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f57a7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a0155df0 0x7f57a00a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f57a7400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a014db20 con 0x7f57a0155a20 2024-08-17T02:45:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f57a7400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a015c1d0 con 0x7f57a0155df0 2024-08-17T02:45:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f57a7400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a015c4b0 con 0x7f57a006df00 2024-08-17T02:45:25.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f5797e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a0155a20 0x7f57a015f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f5797e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a0155a20 0x7f57a015f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57862/0 (socket says 172.21.15.16:57862) 2024-08-17T02:45:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f5797e00640 1 -- 172.21.15.16:0/1792047258 learned_addr learned my addr 172.21.15.16:0/1792047258 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f57a4a00640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a006df00 0x7f57a014d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f57a5400640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a0155df0 0x7f57a00a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:25.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f5797e00640 1 -- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a006df00 msgr2=0x7f57a014d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f5797e00640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a006df00 0x7f57a014d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f5797e00640 1 -- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a0155df0 msgr2=0x7f57a00a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f5797e00640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a0155df0 0x7f57a00a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.617+0000 7f5797e00640 1 -- 172.21.15.16:0/1792047258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5798009ae0 con 0x7f57a0155a20 2024-08-17T02:45:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.618+0000 7f5797e00640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a0155a20 0x7f57a015f680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f578800b4c0 tx=0x7f5788009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.618+0000 7f579cc00640 1 -- 172.21.15.16:0/1792047258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578800f510 con 0x7f57a0155a20 2024-08-17T02:45:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.618+0000 7f579cc00640 1 -- 172.21.15.16:0/1792047258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578800f670 con 0x7f57a0155a20 2024-08-17T02:45:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.618+0000 7f579cc00640 1 -- 172.21.15.16:0/1792047258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578800fa30 con 0x7f57a0155a20 2024-08-17T02:45:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.618+0000 7f57a7400640 1 -- 172.21.15.16:0/1792047258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a015c790 con 0x7f57a0155a20 2024-08-17T02:45:25.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.618+0000 7f57a7400640 1 -- 172.21.15.16:0/1792047258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a015cd50 con 0x7f57a0155a20 2024-08-17T02:45:25.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.619+0000 7f57a7400640 1 -- 172.21.15.16:0/1792047258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a015cf70 con 0x7f57a0155a20 2024-08-17T02:45:25.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.620+0000 7f579cc00640 1 -- 172.21.15.16:0/1792047258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f578801a400 con 0x7f57a0155a20 2024-08-17T02:45:25.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.621+0000 7f579cc00640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f577c065150 0x7f577c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:25.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.621+0000 7f579cc00640 1 -- 172.21.15.16:0/1792047258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5788081ff0 con 0x7f57a0155a20 2024-08-17T02:45:25.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.621+0000 7f57a4a00640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f577c065150 0x7f577c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:25.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.622+0000 7f57a4a00640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f577c065150 0x7f577c067600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f579000bb40 tx=0x7f5790008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.625+0000 7f579cc00640 1 -- 172.21.15.16:0/1792047258 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f57880500f0 con 0x7f57a0155a20 2024-08-17T02:45:25.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.806+0000 7f57a7400640 1 -- 172.21.15.16:0/1792047258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57a0064db0 con 0x7f57a0155a20 2024-08-17T02:45:25.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.807+0000 7f579cc00640 1 -- 172.21.15.16:0/1792047258 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f5788053710 con 0x7f57a0155a20 2024-08-17T02:45:25.808 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:25.809 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.809+0000 7f57a7400640 1 -- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f577c065150 msgr2=0x7f577c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.809+0000 7f57a7400640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f577c065150 0x7f577c067600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f579000bb40 tx=0x7f5790008040 comp rx=0 tx=0).stop 2024-08-17T02:45:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.809+0000 7f57a7400640 1 -- 172.21.15.16:0/1792047258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a0155a20 msgr2=0x7f57a015f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.809+0000 7f57a7400640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a0155a20 0x7f57a015f680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f578800b4c0 tx=0x7f5788009060 comp rx=0 tx=0).stop 2024-08-17T02:45:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.809+0000 7f57a7400640 1 -- 172.21.15.16:0/1792047258 shutdown_connections 2024-08-17T02:45:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.809+0000 7f57a7400640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f577c065150 0x7f577c067600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.809+0000 7f57a7400640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57a0155df0 0x7f57a00a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.809+0000 7f57a7400640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f57a0155a20 0x7f57a015f680 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.809+0000 7f57a7400640 1 --2- 172.21.15.16:0/1792047258 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f57a006df00 0x7f57a014d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.809+0000 7f57a7400640 1 -- 172.21.15.16:0/1792047258 >> 172.21.15.16:0/1792047258 conn(0x7f57a00cb9b0 msgr2=0x7f57a0153750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.809+0000 7f57a7400640 1 -- 172.21.15.16:0/1792047258 shutdown_connections 2024-08-17T02:45:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:25.809+0000 7f57a7400640 1 -- 172.21.15.16:0/1792047258 wait complete. 2024-08-17T02:45:25.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-08-17T02:45:25.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-08-17T02:45:25.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-08-17T02:45:25.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-08-17T02:45:25.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-08-17T02:45:25.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-08-17T02:45:25.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-08-17T02:45:25.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-08-17T02:45:26.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.092+0000 7fc1a5000640 1 Processor -- start 2024-08-17T02:45:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.092+0000 7fc1a5000640 1 -- start start 2024-08-17T02:45:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.094+0000 7fc1a5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1980dfd70 0x7fc1980e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.094+0000 7fc1a5000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc1980e40c0 0x7fc19800d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.094+0000 7fc1a5000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc198054e80 con 0x7fc1980e0140 2024-08-17T02:45:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.094+0000 7fc1a5000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1980e4590 con 0x7fc1980e40c0 2024-08-17T02:45:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.094+0000 7fc1a5000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980e4870 con 0x7fc1980dfd70 2024-08-17T02:45:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.096+0000 7fc19d400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc1980e40c0 0x7fc19800d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.096+0000 7fc19d400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc1980e40c0 0x7fc19800d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36530/0 (socket says 172.21.15.16:36530) 2024-08-17T02:45:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.096+0000 7fc19d400640 1 -- 172.21.15.16:0/2905248464 learned_addr learned my addr 172.21.15.16:0/2905248464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.096+0000 7fc19de00640 1 --2- 172.21.15.16:0/2905248464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1980dfd70 0x7fc1980e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.097+0000 7fc19d400640 1 -- 172.21.15.16:0/2905248464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1980dfd70 msgr2=0x7fc1980e3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.097+0000 7fc19d400640 1 --2- 172.21.15.16:0/2905248464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1980dfd70 0x7fc1980e3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.097+0000 7fc19d400640 1 -- 172.21.15.16:0/2905248464 >> v1:172.21.15.142:6789/0 conn(0x7fc1980e0140 legacy=0x7fc1980358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.097+0000 7fc19d400640 1 -- 172.21.15.16:0/2905248464 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc198054b10 con 0x7fc1980e40c0 2024-08-17T02:45:26.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.099+0000 7f4ba7e00640 1 Processor -- start 2024-08-17T02:45:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.099+0000 7fc19d400640 1 --2- 172.21.15.16:0/2905248464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1980e40c0 0x7fc19800d8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009910 tx=0x7fc18c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5881bb52785cbebb server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.099+0000 7f4ba7e00640 1 -- start start 2024-08-17T02:45:26.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.100+0000 7fc19ca00640 1 -- 172.21.15.16:0/2905248464 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c02add0 con 0x7fc1980e40c0 2024-08-17T02:45:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.100+0000 7fc19ca00640 1 -- 172.21.15.16:0/2905248464 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c032440 con 0x7fc1980e40c0 2024-08-17T02:45:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.100+0000 7fc19ca00640 1 -- 172.21.15.16:0/2905248464 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c032820 con 0x7fc1980e40c0 2024-08-17T02:45:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.101+0000 7fc1a5000640 1 -- 172.21.15.16:0/2905248464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1980e40c0 msgr2=0x7fc19800d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.101+0000 7fc1a5000640 1 --2- 172.21.15.16:0/2905248464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1980e40c0 0x7fc19800d8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009910 tx=0x7fc18c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7fc1a5000640 1 -- 172.21.15.16:0/2905248464 shutdown_connections 2024-08-17T02:45:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7fc1a5000640 1 --2- 172.21.15.16:0/2905248464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1980e40c0 0x7fc19800d8f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7fc1a5000640 1 --2- 172.21.15.16:0/2905248464 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1980dfd70 0x7fc1980e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7f4ba7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ba806e570 0x7f4ba81565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7f4ba7e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ba8156130 0x7f4ba8152730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7f4ba7e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4ba81589d0 0x7f4ba80af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7f4ba7e00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba815bd50 con 0x7f4ba8156130 2024-08-17T02:45:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7f4ba7e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba8159340 con 0x7f4ba81589d0 2024-08-17T02:45:26.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7f5f09a00640 1 Processor -- start 2024-08-17T02:45:26.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7f4ba7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8159620 con 0x7f4ba806e570 2024-08-17T02:45:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7fc1a5000640 1 -- 172.21.15.16:0/2905248464 >> 172.21.15.16:0/2905248464 conn(0x7fc198057870 msgr2=0x7fc19800c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7fc1a5000640 1 -- 172.21.15.16:0/2905248464 shutdown_connections 2024-08-17T02:45:26.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.103+0000 7fc1a5000640 1 -- 172.21.15.16:0/2905248464 wait complete. 2024-08-17T02:45:26.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.102+0000 7f5f09a00640 1 -- start start 2024-08-17T02:45:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.103+0000 7f4ba6a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ba806e570 0x7f4ba81565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.103+0000 7f4ba7400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4ba81589d0 0x7f4ba80af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.103+0000 7f4ba7400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4ba81589d0 0x7f4ba80af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36536/0 (socket says 172.21.15.16:36536) 2024-08-17T02:45:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.103+0000 7f4ba7400640 1 -- 172.21.15.16:0/692986471 learned_addr learned my addr 172.21.15.16:0/692986471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.103+0000 7f4ba6000640 1 --2- 172.21.15.16:0/692986471 >> v2:172.21.15.16:3300/0 conn(0x7f4ba8156130 0x7f4ba8152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.104+0000 7f5f09a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5efc057590 0x7f5efc00d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.104+0000 7f5f09a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5efc0e0130 0x7f5efc00dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.104+0000 7f5f09a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc036970 con 0x7f5efc0dfd60 2024-08-17T02:45:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.104+0000 7f5f09a00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5efc035fd0 con 0x7f5efc057590 2024-08-17T02:45:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.104+0000 7f5f09a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5efc054970 con 0x7f5efc0e0130 2024-08-17T02:45:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.105+0000 7f5f02a00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f5efc0dfd60 0x7f5efc0358c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:34918/0 (socket says 172.21.15.16:34918) 2024-08-17T02:45:26.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.105+0000 7f5f02a00640 1 -- 172.21.15.16:0/2646855079 learned_addr learned my addr 172.21.15.16:0/2646855079 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.105+0000 7f5f02000640 1 --2- 172.21.15.16:0/2646855079 >> v2:172.21.15.16:3300/0 conn(0x7f5efc057590 0x7f5efc00d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.106+0000 7f5f03400640 1 --2- 172.21.15.16:0/2646855079 >> v2:172.21.15.142:3300/0 conn(0x7f5efc0e0130 0x7f5efc00dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.105+0000 7f4ba7400640 1 -- 172.21.15.16:0/692986471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ba806e570 msgr2=0x7f4ba81565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.106+0000 7fc1a5000640 1 Processor -- start 2024-08-17T02:45:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.105+0000 7f4ba7400640 1 --2- 172.21.15.16:0/692986471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ba806e570 0x7f4ba81565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.105+0000 7f4ba7400640 1 -- 172.21.15.16:0/692986471 >> v2:172.21.15.16:3300/0 conn(0x7f4ba8156130 msgr2=0x7f4ba8152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.105+0000 7f4ba7400640 1 --2- 172.21.15.16:0/692986471 >> v2:172.21.15.16:3300/0 conn(0x7f4ba8156130 0x7f4ba8152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.105+0000 7f4ba7400640 1 -- 172.21.15.16:0/692986471 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba80aa870 con 0x7f4ba81589d0 2024-08-17T02:45:26.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.106+0000 7fc1a5000640 1 -- start start 2024-08-17T02:45:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.106+0000 7f4ba6000640 1 --2- 172.21.15.16:0/692986471 >> v2:172.21.15.16:3300/0 conn(0x7f4ba8156130 0x7f4ba8152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.107+0000 7f4ba7400640 1 --2- 172.21.15.16:0/692986471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba81589d0 0x7f4ba80af7c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80646a0 tx=0x7f4b980091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=768f56e01ee57e9b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.107+0000 7f5f01600640 1 -- 172.21.15.16:0/2646855079 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315926063 0 0) 0x7f5efc036970 con 0x7f5efc0dfd60 2024-08-17T02:45:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7fc1a5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1980dfd70 0x7fc198009180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7fc1a5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1980e0140 0x7fc1980cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7fc1a5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1980e40c0 0x7fc1980e1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7fc1a5000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980d8950 con 0x7fc1980e40c0 2024-08-17T02:45:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7fc1a5000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc198008cf0 con 0x7fc1980e0140 2024-08-17T02:45:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7fc1a5000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980d00a0 con 0x7fc1980dfd70 2024-08-17T02:45:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.107+0000 7f5f01600640 1 -- 172.21.15.16:0/2646855079 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee80034e0 con 0x7f5efc0dfd60 2024-08-17T02:45:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7f5f03400640 1 -- 172.21.15.16:0/2646855079 >> v2:172.21.15.16:3300/0 conn(0x7f5efc057590 msgr2=0x7f5efc00d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7f5f01600640 1 -- 172.21.15.16:0/2646855079 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3768522665 0 0) 0x7f5ee80034e0 con 0x7f5efc0dfd60 2024-08-17T02:45:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7f4ba5600640 1 -- 172.21.15.16:0/692986471 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9802ae60 con 0x7f4ba81589d0 2024-08-17T02:45:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7f4ba5600640 1 -- 172.21.15.16:0/692986471 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b98032440 con 0x7f4ba81589d0 2024-08-17T02:45:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7f4ba5600640 1 -- 172.21.15.16:0/692986471 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b980327e0 con 0x7f4ba81589d0 2024-08-17T02:45:26.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7fc19de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1980dfd70 0x7fc198009180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7fc19de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1980dfd70 0x7fc198009180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:52910/0 (socket says 172.21.15.16:52910) 2024-08-17T02:45:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7fc19de00640 1 -- 172.21.15.16:0/1367844359 learned_addr learned my addr 172.21.15.16:0/1367844359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.108+0000 7fc19d400640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1980e0140 0x7fc1980cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.109+0000 7fc19d400640 1 -- 172.21.15.16:0/1367844359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1980dfd70 msgr2=0x7fc198009180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.109+0000 7fc19d400640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc1980dfd70 0x7fc198009180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.109+0000 7fc19e800640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1980e40c0 0x7fc1980e1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.109+0000 7fc19d400640 1 -- 172.21.15.16:0/1367844359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1980e40c0 msgr2=0x7fc1980e1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.109+0000 7fc19d400640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1980e40c0 0x7fc1980e1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.109+0000 7fc19d400640 1 -- 172.21.15.16:0/1367844359 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c009450 con 0x7fc1980e0140 2024-08-17T02:45:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.109+0000 7fc19e800640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1980e40c0 0x7fc1980e1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.109+0000 7f5f03400640 1 --2- 172.21.15.16:0/2646855079 >> v2:172.21.15.16:3300/0 conn(0x7f5efc057590 0x7f5efc00d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.109+0000 7f5f03400640 1 -- 172.21.15.16:0/2646855079 >> v1:172.21.15.142:6789/0 conn(0x7f5efc0dfd60 legacy=0x7f5efc0358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.109+0000 7f5f03400640 1 -- 172.21.15.16:0/2646855079 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc054f30 con 0x7f5efc0e0130 2024-08-17T02:45:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f5f02000640 1 --2- 172.21.15.16:0/2646855079 >> v2:172.21.15.16:3300/0 conn(0x7f5efc057590 0x7f5efc00d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7fc19d400640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1980e0140 0x7fc1980cfb60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc19800f250 tx=0x7fc18c032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f5f03400640 1 --2- 172.21.15.16:0/2646855079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5efc0e0130 0x7f5efc00dea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f0404ca10 tx=0x7f5f04064990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b6e6855ed4f8f3e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f4ba7e00640 1 -- 172.21.15.16:0/692986471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba81589d0 msgr2=0x7f4ba80af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f4ba7e00640 1 --2- 172.21.15.16:0/692986471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba81589d0 0x7f4ba80af7c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80646a0 tx=0x7f4b980091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7fedb3400640 1 Processor -- start 2024-08-17T02:45:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f4ba7e00640 1 -- 172.21.15.16:0/692986471 shutdown_connections 2024-08-17T02:45:26.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f5f01600640 1 -- 172.21.15.16:0/2646855079 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4001470 con 0x7f5efc0e0130 2024-08-17T02:45:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f4ba7e00640 1 --2- 172.21.15.16:0/692986471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba81589d0 0x7f4ba80af7c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7fc195600640 1 -- 172.21.15.16:0/1367844359 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c000d10 con 0x7fc1980e0140 2024-08-17T02:45:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7fc195600640 1 -- 172.21.15.16:0/1367844359 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c031a60 con 0x7fc1980e0140 2024-08-17T02:45:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7fc195600640 1 -- 172.21.15.16:0/1367844359 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c031e60 con 0x7fc1980e0140 2024-08-17T02:45:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f5f01600640 1 -- 172.21.15.16:0/2646855079 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f040a4050 con 0x7f5efc0e0130 2024-08-17T02:45:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.111+0000 7f5f01600640 1 -- 172.21.15.16:0/2646855079 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f040a8610 con 0x7f5efc0e0130 2024-08-17T02:45:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f4ba7e00640 1 --2- 172.21.15.16:0/692986471 >> v2:172.21.15.16:3300/0 conn(0x7f4ba8156130 0x7f4ba8152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f4ba7e00640 1 --2- 172.21.15.16:0/692986471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ba806e570 0x7f4ba81565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f4ba7e00640 1 -- 172.21.15.16:0/692986471 >> 172.21.15.16:0/692986471 conn(0x7f4ba80cb790 msgr2=0x7f4ba80cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f4ba7e00640 1 -- 172.21.15.16:0/692986471 shutdown_connections 2024-08-17T02:45:26.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.110+0000 7f4ba7e00640 1 -- 172.21.15.16:0/692986471 wait complete. 2024-08-17T02:45:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.111+0000 7fc1a5000640 1 -- 172.21.15.16:0/1367844359 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1980080e0 con 0x7fc1980e0140 2024-08-17T02:45:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.111+0000 7fc1a5000640 1 -- 172.21.15.16:0/1367844359 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1980086a0 con 0x7fc1980e0140 2024-08-17T02:45:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.112+0000 7f5f09a00640 1 -- 172.21.15.16:0/2646855079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5efc0e0130 msgr2=0x7f5efc00dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.112+0000 7f5f09a00640 1 --2- 172.21.15.16:0/2646855079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5efc0e0130 0x7f5efc00dea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f0404ca10 tx=0x7f5f04064990 comp rx=0 tx=0).stop 2024-08-17T02:45:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.112+0000 7f5f09a00640 1 -- 172.21.15.16:0/2646855079 shutdown_connections 2024-08-17T02:45:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.112+0000 7f5f09a00640 1 --2- 172.21.15.16:0/2646855079 >> v2:172.21.15.16:3300/0 conn(0x7f5efc057590 0x7f5efc00d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.112+0000 7f5f09a00640 1 --2- 172.21.15.16:0/2646855079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5efc0e0130 0x7f5efc00dea0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.112+0000 7f5f09a00640 1 -- 172.21.15.16:0/2646855079 >> 172.21.15.16:0/2646855079 conn(0x7f5efc00a830 msgr2=0x7f5efc056da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.111+0000 7fedb3400640 1 -- start start 2024-08-17T02:45:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.113+0000 7f5f09a00640 1 -- 172.21.15.16:0/2646855079 shutdown_connections 2024-08-17T02:45:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.112+0000 7fc1a5000640 1 -- 172.21.15.16:0/1367844359 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc198005770 con 0x7fc1980e0140 2024-08-17T02:45:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.113+0000 7f4ba7e00640 1 Processor -- start 2024-08-17T02:45:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.113+0000 7f4ba7e00640 1 -- start start 2024-08-17T02:45:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.113+0000 7f5f09a00640 1 -- 172.21.15.16:0/2646855079 wait complete. 2024-08-17T02:45:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.114+0000 7fedb3400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fedac0b0610 0x7fedac15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.114+0000 7fedb3400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedac15e140 0x7fedac166e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.114+0000 7fedb3400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedac0b1690 con 0x7fedac15d830 2024-08-17T02:45:26.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.114+0000 7fedb3400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedac06e8c0 con 0x7fedac0b0610 2024-08-17T02:45:26.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.114+0000 7fedb3400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0b1b20 con 0x7fedac15e140 2024-08-17T02:45:26.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.114+0000 7f4ba7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba806e570 0x7f4ba8144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.114+0000 7f4ba7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba8156130 0x7f4ba813c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.115+0000 7fedb0a00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fedac15d830 0x7fedac06e1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:34926/0 (socket says 172.21.15.16:34926) 2024-08-17T02:45:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.115+0000 7fedb0a00640 1 -- 172.21.15.16:0/4065255499 learned_addr learned my addr 172.21.15.16:0/4065255499 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.114+0000 7f4ba7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ba81589d0 0x7f4ba8142800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.114+0000 7f4ba7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba81548a0 con 0x7f4ba8156130 2024-08-17T02:45:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.114+0000 7f4ba7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8153c90 con 0x7f4ba806e570 2024-08-17T02:45:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.114+0000 7f4ba7e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8153f70 con 0x7f4ba81589d0 2024-08-17T02:45:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.115+0000 7f4ba7400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ba81589d0 0x7f4ba8142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.115+0000 7fedb1400640 1 --2- 172.21.15.16:0/4065255499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedac15e140 0x7fedac166e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.116+0000 7f5f09a00640 1 Processor -- start 2024-08-17T02:45:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.115+0000 7f4ba6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba806e570 0x7f4ba8144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.115+0000 7f4ba7400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ba81589d0 0x7f4ba8142800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:52916/0 (socket says 172.21.15.16:52916) 2024-08-17T02:45:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.115+0000 7f4ba7400640 1 -- 172.21.15.16:0/1972952161 learned_addr learned my addr 172.21.15.16:0/1972952161 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.115+0000 7f4ba6000640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba8156130 0x7f4ba813c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.116+0000 7fc195600640 1 -- 172.21.15.16:0/1367844359 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fc18c039040 con 0x7fc1980e0140 2024-08-17T02:45:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.116+0000 7f5f09a00640 1 -- start start 2024-08-17T02:45:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.116+0000 7f4ba6a00640 1 -- 172.21.15.16:0/1972952161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ba81589d0 msgr2=0x7f4ba8142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.116+0000 7f4ba6a00640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ba81589d0 0x7f4ba8142800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.116+0000 7f4ba6a00640 1 -- 172.21.15.16:0/1972952161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba8156130 msgr2=0x7f4ba813c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.116+0000 7f4ba6a00640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba8156130 0x7f4ba813c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.116+0000 7f4ba6a00640 1 -- 172.21.15.16:0/1972952161 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b94009390 con 0x7f4ba806e570 2024-08-17T02:45:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.117+0000 7f4ba7400640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ba81589d0 0x7f4ba8142800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.116+0000 7f1a43000640 1 Processor -- start 2024-08-17T02:45:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.116+0000 7fedb1400640 1 -- 172.21.15.16:0/4065255499 >> v2:172.21.15.142:3300/0 conn(0x7fedac0b0610 msgr2=0x7fedac15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.116+0000 7fedab400640 1 -- 172.21.15.16:0/4065255499 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000617830 0 0) 0x7fedac0b1690 con 0x7fedac15d830 2024-08-17T02:45:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.117+0000 7f4ba6a00640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba806e570 0x7f4ba8144c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000ed60 tx=0x7f4ba000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.117+0000 7fedabe00640 1 --2- 172.21.15.16:0/4065255499 >> v2:172.21.15.142:3300/0 conn(0x7fedac0b0610 0x7fedac15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.117+0000 7fc195600640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc180065220 0x7fc1800676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.117+0000 7fc195600640 1 -- 172.21.15.16:0/1367844359 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc18c041070 con 0x7fc1980e0140 2024-08-17T02:45:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.117+0000 7fedb1400640 1 --2- 172.21.15.16:0/4065255499 >> v2:172.21.15.142:3300/0 conn(0x7fedac0b0610 0x7fedac15dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.117+0000 7fedb1400640 1 -- 172.21.15.16:0/4065255499 >> v1:172.21.15.142:6789/0 conn(0x7fedac15d830 legacy=0x7fedac06e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.117+0000 7fedb1400640 1 -- 172.21.15.16:0/4065255499 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac0ae4a0 con 0x7fedac15e140 2024-08-17T02:45:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.116+0000 7f1a43000640 1 -- start start 2024-08-17T02:45:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7fc195600640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] conn(0x7fc18006d2c0 0x7fc18006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7fc195600640 1 -- 172.21.15.16:0/1367844359 --> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc18006fd80 con 0x7fc18006d2c0 2024-08-17T02:45:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7fc195600640 1 -- 172.21.15.16:0/1367844359 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc18c0a2e30 con 0x7fc1980e0140 2024-08-17T02:45:26.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7f5f09a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5efc057590 0x7f5efc037fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7f5f09a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5efc0dfd60 0x7f5efc038520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7f5f09a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5efc0e0130 0x7f5efc039d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7f5f09a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc039020 con 0x7f5efc0e0130 2024-08-17T02:45:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7f5f09a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0dbf50 con 0x7f5efc0dfd60 2024-08-17T02:45:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7f5f09a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc04f7b0 con 0x7f5efc057590 2024-08-17T02:45:26.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7fc19de00640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc180065220 0x7fc1800676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7f4b9e000640 1 -- 172.21.15.16:0/1972952161 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000eec0 con 0x7f4ba806e570 2024-08-17T02:45:26.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7f4b9e000640 1 -- 172.21.15.16:0/1972952161 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba000f040 con 0x7f4ba806e570 2024-08-17T02:45:26.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7f4b9e000640 1 -- 172.21.15.16:0/1972952161 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0014480 con 0x7f4ba806e570 2024-08-17T02:45:26.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.118+0000 7fc19e800640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] conn(0x7fc18006d2c0 0x7fc18006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f1a43000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a3c0c6250 0x7f1a3c0c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f1a43000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1a3c15d610 0x7f1a3c0c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f1a43000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3c0b4470 con 0x7f1a3c0b4020 2024-08-17T02:45:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f1a43000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3c0b4700 con 0x7f1a3c15d610 2024-08-17T02:45:26.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f1a43000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c0c4c20 con 0x7f1a3c0c6250 2024-08-17T02:45:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f4ba7e00640 1 -- 172.21.15.16:0/1972952161 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b98009450 con 0x7f4ba806e570 2024-08-17T02:45:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f4ba7e00640 1 -- 172.21.15.16:0/1972952161 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba8154530 con 0x7f4ba806e570 2024-08-17T02:45:26.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7fc19e800640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] conn(0x7fc18006d2c0 0x7fc18006f6d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f5f02000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5efc0dfd60 0x7f5efc038520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f5f02000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5efc0dfd60 0x7f5efc038520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36588/0 (socket says 172.21.15.16:36588) 2024-08-17T02:45:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f5f02000640 1 -- 172.21.15.16:0/3707657471 learned_addr learned my addr 172.21.15.16:0/3707657471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f1a3be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a3c0c6250 0x7f1a3c0c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f1a3be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a3c0c6250 0x7f1a3c0c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:52934/0 (socket says 172.21.15.16:52934) 2024-08-17T02:45:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.119+0000 7f1a3be00640 1 -- 172.21.15.16:0/254826935 learned_addr learned my addr 172.21.15.16:0/254826935 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.120+0000 7f1a41000640 1 --2- 172.21.15.16:0/254826935 >> v2:172.21.15.142:3300/0 conn(0x7f1a3c15d610 0x7f1a3c0c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.120+0000 7fedb1400640 1 --2- 172.21.15.16:0/4065255499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedac15e140 0x7fedac166e50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feda4009e60 tx=0x7feda4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=558585460bccb68e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.120+0000 7f5f03400640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5efc0e0130 0x7f5efc039d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.120+0000 7fc19de00640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc180065220 0x7fc1800676d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc190007c60 tx=0x7fc190008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7fc195600640 1 -- 172.21.15.16:0/1367844359 <== osd.0 v2:172.21.15.16:6824/37182 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fc18006fd80 con 0x7fc18006d2c0 2024-08-17T02:45:26.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7fedab400640 1 -- 172.21.15.16:0/4065255499 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4007550 con 0x7fedac15e140 2024-08-17T02:45:26.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7f5f02000640 1 -- 172.21.15.16:0/3707657471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5efc057590 msgr2=0x7f5efc037fe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-17T02:45:26.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7f5f02000640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5efc057590 0x7f5efc037fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7f5f02000640 1 -- 172.21.15.16:0/3707657471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5efc0e0130 msgr2=0x7f5efc039d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7f5f02000640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5efc0e0130 0x7f5efc039d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7f5f02000640 1 -- 172.21.15.16:0/3707657471 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef0009390 con 0x7f5efc0dfd60 2024-08-17T02:45:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7f1a41000640 1 -- 172.21.15.16:0/254826935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a3c0c6250 msgr2=0x7f1a3c0c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7f1a41000640 1 --2- 172.21.15.16:0/254826935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a3c0c6250 0x7f1a3c0c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7f1a41000640 1 -- 172.21.15.16:0/254826935 >> v1:172.21.15.142:6789/0 conn(0x7f1a3c0b4020 legacy=0x7f1a3c15cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7f1a41000640 1 -- 172.21.15.16:0/254826935 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c0c4f00 con 0x7f1a3c15d610 2024-08-17T02:45:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7f1a3aa00640 1 -- 172.21.15.16:0/254826935 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819184148 0 0) 0x7f1a3c0b4470 con 0x7f1a3c0b4020 2024-08-17T02:45:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7ffa40a00640 1 Processor -- start 2024-08-17T02:45:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7fedab400640 1 -- 172.21.15.16:0/4065255499 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda402d050 con 0x7fedac15e140 2024-08-17T02:45:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7fedab400640 1 -- 172.21.15.16:0/4065255499 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda40078e0 con 0x7fedac15e140 2024-08-17T02:45:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.122+0000 7fedb3400640 1 -- 172.21.15.16:0/4065255499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedac15e140 msgr2=0x7fedac166e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.122+0000 7fedb3400640 1 --2- 172.21.15.16:0/4065255499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedac15e140 0x7fedac166e50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feda4009e60 tx=0x7feda4009780 comp rx=0 tx=0).stop 2024-08-17T02:45:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.121+0000 7f4ba7e00640 1 -- 172.21.15.16:0/1972952161 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4ba8071050 con 0x7f4ba806e570 2024-08-17T02:45:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.122+0000 7fedb3400640 1 -- 172.21.15.16:0/4065255499 shutdown_connections 2024-08-17T02:45:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.122+0000 7fedb3400640 1 --2- 172.21.15.16:0/4065255499 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedac15e140 0x7fedac166e50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.122+0000 7fedb3400640 1 --2- 172.21.15.16:0/4065255499 >> v2:172.21.15.142:3300/0 conn(0x7fedac0b0610 0x7fedac15dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.122+0000 7fedb3400640 1 -- 172.21.15.16:0/4065255499 >> 172.21.15.16:0/4065255499 conn(0x7fedac0cb9b0 msgr2=0x7fedac0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.122+0000 7fedb3400640 1 -- 172.21.15.16:0/4065255499 shutdown_connections 2024-08-17T02:45:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.122+0000 7fedb3400640 1 -- 172.21.15.16:0/4065255499 wait complete. 2024-08-17T02:45:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.122+0000 7f5f02000640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5efc0dfd60 0x7f5efc038520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000ec80 tx=0x7f5ef00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.123+0000 7f5efa000640 1 -- 172.21.15.16:0/3707657471 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef000eec0 con 0x7f5efc0dfd60 2024-08-17T02:45:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.123+0000 7f5efa000640 1 -- 172.21.15.16:0/3707657471 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef000cbc0 con 0x7f5efc0dfd60 2024-08-17T02:45:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.123+0000 7f5efa000640 1 -- 172.21.15.16:0/3707657471 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0013470 con 0x7f5efc0dfd60 2024-08-17T02:45:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.123+0000 7f4b9e000640 1 -- 172.21.15.16:0/1972952161 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f4ba0014870 con 0x7f4ba806e570 2024-08-17T02:45:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.122+0000 7ffa40a00640 1 -- start start 2024-08-17T02:45:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.124+0000 7ffa40a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa3c158c80 0x7ffa3c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.124+0000 7ffa40a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa3c0cd700 0x7ffa3c0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.124+0000 7ffa40a00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa3c0cf9d0 con 0x7ffa3c0cc800 2024-08-17T02:45:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.124+0000 7ffa40a00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa3c0ceb60 con 0x7ffa3c158c80 2024-08-17T02:45:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.124+0000 7ffa40a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa3c0cedc0 con 0x7ffa3c0cd700 2024-08-17T02:45:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7f4b9e000640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4b880652a0 0x7f4b88067750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.124+0000 7f1a41000640 1 --2- 172.21.15.16:0/254826935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a3c15d610 0x7f1a3c0c59e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009850 tx=0x7f1a240090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6765061a1554c3d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7ffa32a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ffa3c0cd700 0x7ffa3c0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7f4b9e000640 1 -- 172.21.15.16:0/1972952161 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ba0082f90 con 0x7f4ba806e570 2024-08-17T02:45:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7f1a3aa00640 1 -- 172.21.15.16:0/254826935 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2402ad90 con 0x7f1a3c15d610 2024-08-17T02:45:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7ffa3be00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffa3c0cc800 0x7ffa3c0ca380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46130/0 (socket says 172.21.15.16:46130) 2024-08-17T02:45:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7ffa3be00640 1 -- 172.21.15.16:0/265474804 learned_addr learned my addr 172.21.15.16:0/265474804 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7f1a3aa00640 1 -- 172.21.15.16:0/254826935 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a24032440 con 0x7f1a3c15d610 2024-08-17T02:45:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.124+0000 7f5f09a00640 1 -- 172.21.15.16:0/3707657471 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f04065e30 con 0x7f5efc0dfd60 2024-08-17T02:45:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.124+0000 7f5f09a00640 1 -- 172.21.15.16:0/3707657471 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc04f160 con 0x7f5efc0dfd60 2024-08-17T02:45:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7f5f09a00640 1 -- 172.21.15.16:0/3707657471 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5efc005770 con 0x7f5efc0dfd60 2024-08-17T02:45:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7ffa3b400640 1 --2- 172.21.15.16:0/265474804 >> v2:172.21.15.16:3300/0 conn(0x7ffa3c158c80 0x7ffa3c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.126+0000 7fedb3400640 1 Processor -- start 2024-08-17T02:45:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7efdb0200640 1 Processor -- start 2024-08-17T02:45:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.126+0000 7efdb0200640 1 -- start start 2024-08-17T02:45:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7f4b9e000640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] conn(0x7f4b8806d4b0 0x7f4b8806f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7f4b9e000640 1 -- 172.21.15.16:0/1972952161 --> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b8806ff70 con 0x7f4b8806d4b0 2024-08-17T02:45:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.125+0000 7f4b9e000640 1 -- 172.21.15.16:0/1972952161 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ba00854d0 con 0x7f4ba806e570 2024-08-17T02:45:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.126+0000 7f4ba7400640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] conn(0x7f4b8806d4b0 0x7f4b8806f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.126+0000 7f4ba6000640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4b880652a0 0x7f4b88067750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.126+0000 7f1a3aa00640 1 -- 172.21.15.16:0/254826935 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2402ad90 con 0x7f1a3c15d610 2024-08-17T02:45:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.126+0000 7f1a43000640 1 -- 172.21.15.16:0/254826935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a3c15d610 msgr2=0x7f1a3c0c59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.126+0000 7f1a43000640 1 --2- 172.21.15.16:0/254826935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a3c15d610 0x7f1a3c0c59e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009850 tx=0x7f1a240090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.126+0000 7fedb3400640 1 -- start start 2024-08-17T02:45:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.126+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074064610 0 0) 0x7ffa3c0cf9d0 con 0x7ffa3c0cc800 2024-08-17T02:45:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.126+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa200034e0 con 0x7ffa3c0cc800 2024-08-17T02:45:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7f1a43000640 1 -- 172.21.15.16:0/254826935 shutdown_connections 2024-08-17T02:45:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7f1a43000640 1 --2- 172.21.15.16:0/254826935 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a3c15d610 0x7f1a3c0c59e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7f1a43000640 1 --2- 172.21.15.16:0/254826935 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a3c0c6250 0x7f1a3c0c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7f1a43000640 1 -- 172.21.15.16:0/254826935 >> 172.21.15.16:0/254826935 conn(0x7f1a3c0cfde0 msgr2=0x7f1a3c0d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2252772143 0 0) 0x7ffa200034e0 con 0x7ffa3c0cc800 2024-08-17T02:45:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 >> v2:172.21.15.142:3300/0 conn(0x7ffa3c0cd700 msgr2=0x7ffa3c0cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7ffa3aa00640 1 --2- 172.21.15.16:0/265474804 >> v2:172.21.15.142:3300/0 conn(0x7ffa3c0cd700 0x7ffa3c0cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 >> v2:172.21.15.16:3300/0 conn(0x7ffa3c158c80 msgr2=0x7ffa3c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7ffa3aa00640 1 --2- 172.21.15.16:0/265474804 >> v2:172.21.15.16:3300/0 conn(0x7ffa3c158c80 0x7ffa3c0ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c0cf0a0 con 0x7ffa3c0cc800 2024-08-17T02:45:26.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7ffa34001470 con 0x7ffa3c0cc800 2024-08-17T02:45:26.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 >> v1:172.21.15.16:6789/0 conn(0x7ffa3c0cc800 legacy=0x7ffa3c0ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7f1a43000640 1 -- 172.21.15.16:0/254826935 shutdown_connections 2024-08-17T02:45:26.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7f4ba7400640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] conn(0x7f4b8806d4b0 0x7f4b8806f8c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7ffa3aa00640 1 --2- 172.21.15.16:0/265474804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200039d0 0x7ffa20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7f1a43000640 1 -- 172.21.15.16:0/254826935 wait complete. 2024-08-17T02:45:26.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7fedb3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedac0b0610 0x7fedac0a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.127+0000 7ffa3aa00640 1 --2- 172.21.15.16:0/265474804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa20004780 0x7ffa20004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7ffa3aa00640 1 --2- 172.21.15.16:0/265474804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20005130 0x7ffa2000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c0ceb60 con 0x7ffa20005130 2024-08-17T02:45:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c0cedc0 con 0x7ffa200039d0 2024-08-17T02:45:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2000bf90 con 0x7ffa20004780 2024-08-17T02:45:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7ffa3be00640 1 --2- 172.21.15.16:0/265474804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20005130 0x7ffa2000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7fedb3400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedac15d830 0x7fedac1614e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7f054de00640 1 Processor -- start 2024-08-17T02:45:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7fedb3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac15e140 0x7fedac155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7fedb3400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0a7990 con 0x7fedac15e140 2024-08-17T02:45:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7fedb1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac15e140 0x7fedac155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7fedb1400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac15e140 0x7fedac155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57920/0 (socket says 172.21.15.16:57920) 2024-08-17T02:45:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7fedb1400640 1 -- 172.21.15.16:0/2372119660 learned_addr learned my addr 172.21.15.16:0/2372119660 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7fedb0a00640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedac0b0610 0x7fedac0a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.128+0000 7f054de00640 1 -- start start 2024-08-17T02:45:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac1564b0 con 0x7fedac0b0610 2024-08-17T02:45:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0a6d80 con 0x7fedac15d830 2024-08-17T02:45:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7fedabe00640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedac15d830 0x7fedac1614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7ffa3b400640 1 --2- 172.21.15.16:0/265474804 >> v2:172.21.15.16:3300/0 conn(0x7ffa3c158c80 0x7ffa3c0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7f4ba6000640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4b880652a0 0x7f4b88067750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80cc4d0 tx=0x7f4b940090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7fedb1400640 1 -- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedac15d830 msgr2=0x7fedac1614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7ffa3be00640 1 -- 172.21.15.16:0/265474804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa20004780 msgr2=0x7ffa20004bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-17T02:45:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7ffa3be00640 1 --2- 172.21.15.16:0/265474804 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa20004780 0x7ffa20004bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7ffa3be00640 1 -- 172.21.15.16:0/265474804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200039d0 msgr2=0x7ffa20003de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-17T02:45:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7f054de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548159060 0x7f05480c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7f054de00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f05480c6a30 0x7f05480d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7f054de00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05480cbf40 con 0x7f0548159430 2024-08-17T02:45:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7f054de00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05480cb930 con 0x7f05480c6a30 2024-08-17T02:45:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7f054de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05480ced10 con 0x7f0548159060 2024-08-17T02:45:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7efdb0200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7efda80c9200 0x7efda80b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7efdb0200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efda80b1270 0x7efda80abc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7efdb0200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda80b1690 con 0x7efda81403f0 2024-08-17T02:45:26.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7efdb0200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda80c9660 con 0x7efda80b1270 2024-08-17T02:45:26.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.129+0000 7efdb0200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda80caa00 con 0x7efda80c9200 2024-08-17T02:45:26.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7fedb1400640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fedac15d830 0x7fedac1614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7fedb1400640 1 -- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedac0b0610 msgr2=0x7fedac0a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7fedb1400640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedac0b0610 0x7fedac0a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7fedb1400640 1 -- 172.21.15.16:0/2372119660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda4009a20 con 0x7fedac15e140 2024-08-17T02:45:26.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7f1a43000640 1 Processor -- start 2024-08-17T02:45:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7f5efa000640 1 -- 172.21.15.16:0/3707657471 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f5ef00137e0 con 0x7f5efc0dfd60 2024-08-17T02:45:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7f0546000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f05480c6a30 0x7f05480d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7f0546000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f05480c6a30 0x7f05480d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36632/0 (socket says 172.21.15.16:36632) 2024-08-17T02:45:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7f0546000640 1 -- 172.21.15.16:0/1393137947 learned_addr learned my addr 172.21.15.16:0/1393137947 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7f0546a00640 1 --2- 172.21.15.16:0/1393137947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548159060 0x7f05480c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7f5efa000640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ee40652a0 0x7f5ee4067750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7fedb1400640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac15e140 0x7fedac155f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feda40099f0 tx=0x7feda4037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.130+0000 7ffa3be00640 1 --2- 172.21.15.16:0/265474804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa200039d0 0x7ffa20003de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7ffa3be00640 1 -- 172.21.15.16:0/265474804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c0cf0a0 con 0x7ffa20005130 2024-08-17T02:45:26.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7f0546000640 1 -- 172.21.15.16:0/1393137947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548159060 msgr2=0x7f05480c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7f0546000640 1 --2- 172.21.15.16:0/1393137947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548159060 0x7f05480c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7f0546000640 1 -- 172.21.15.16:0/1393137947 >> v1:172.21.15.142:6789/0 conn(0x7f0548159430 legacy=0x7f05480cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7efdad800640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efda81403f0 0x7efda80ca220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46136/0 (socket says 172.21.15.16:46136) 2024-08-17T02:45:26.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7efdad800640 1 -- 172.21.15.16:0/1799905923 learned_addr learned my addr 172.21.15.16:0/1799905923 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:26.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7efdace00640 1 --2- 172.21.15.16:0/1799905923 >> v2:172.21.15.142:3300/0 conn(0x7efda80c9200 0x7efda80b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7f0546000640 1 -- 172.21.15.16:0/1393137947 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05480cbbd0 con 0x7f05480c6a30 2024-08-17T02:45:26.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7f0546a00640 1 --2- 172.21.15.16:0/1393137947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548159060 0x7f05480c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:26.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7f1a43000640 1 -- start start 2024-08-17T02:45:26.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.131+0000 7f5efa000640 1 -- 172.21.15.16:0/3707657471 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5ef0082b10 con 0x7f5efc0dfd60 2024-08-17T02:45:26.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.132+0000 7f5f02a00640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ee40652a0 0x7f5ee4067750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.132+0000 7ffa3be00640 1 -- 172.21.15.16:0/265474804 reap_dead start 2024-08-17T02:45:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.132+0000 7f5efa000640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] conn(0x7f5ee406d4b0 0x7f5ee406f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.132+0000 7efdace00640 1 -- 172.21.15.16:0/1799905923 >> v2:172.21.15.16:3300/0 conn(0x7efda80b1270 msgr2=0x7efda80abc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:45:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.132+0000 7f4b9e000640 1 -- 172.21.15.16:0/1972952161 <== osd.2 v2:172.21.15.16:6816/37183 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f4b8806ff70 con 0x7f4b8806d4b0 2024-08-17T02:45:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.132+0000 7fed9fe00640 1 -- 172.21.15.16:0/2372119660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4031c10 con 0x7fedac15e140 2024-08-17T02:45:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.132+0000 7fed9fe00640 1 -- 172.21.15.16:0/2372119660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda4031d70 con 0x7fedac15e140 2024-08-17T02:45:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.132+0000 7fed9fe00640 1 -- 172.21.15.16:0/2372119660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda403c630 con 0x7fedac15e140 2024-08-17T02:45:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.132+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac0a7060 con 0x7fedac15e140 2024-08-17T02:45:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.132+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac0a7620 con 0x7fedac15e140 2024-08-17T02:45:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.132+0000 7ffa3be00640 1 --2- 172.21.15.16:0/265474804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20005130 0x7ffa2000b9d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400c410 tx=0x7ffa34007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cca27efe4e11336 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.132+0000 7f5efa000640 1 -- 172.21.15.16:0/3707657471 --> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ee406ff70 con 0x7f5ee406d4b0 2024-08-17T02:45:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.133+0000 7f5efa000640 1 -- 172.21.15.16:0/3707657471 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ef0087220 con 0x7f5efc0dfd60 2024-08-17T02:45:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.133+0000 7f5f03400640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] conn(0x7f5ee406d4b0 0x7f5ee406f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.133+0000 7f5f02a00640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ee40652a0 0x7f5ee4067750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4007a10 tx=0x7f5ef4001540 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.133+0000 7f0546000640 1 --2- 172.21.15.16:0/1393137947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05480c6a30 0x7f05480d08f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0534009910 tx=0x7f05340091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63f34eb4e59ccc71 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.133+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fedac064db0 con 0x7fedac15e140 2024-08-17T02:45:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.134+0000 7f0545600640 1 -- 172.21.15.16:0/1393137947 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053402ad30 con 0x7f05480c6a30 2024-08-17T02:45:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.134+0000 7f0545600640 1 -- 172.21.15.16:0/1393137947 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053402ae90 con 0x7f05480c6a30 2024-08-17T02:45:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.134+0000 7f0545600640 1 -- 172.21.15.16:0/1393137947 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534032670 con 0x7f05480c6a30 2024-08-17T02:45:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.134+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34005890 con 0x7ffa20005130 2024-08-17T02:45:26.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.134+0000 7ffa3aa00640 1 -- 172.21.15.16:0/265474804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa34005a10 con 0x7ffa20005130 2024-08-17T02:45:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.134+0000 7f054de00640 1 -- 172.21.15.16:0/1393137947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05480c6a30 msgr2=0x7f05480d08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.134+0000 7f054de00640 1 --2- 172.21.15.16:0/1393137947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05480c6a30 0x7f05480d08f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0534009910 tx=0x7f05340091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.134+0000 7f054de00640 1 -- 172.21.15.16:0/1393137947 shutdown_connections 2024-08-17T02:45:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.134+0000 7f054de00640 1 --2- 172.21.15.16:0/1393137947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05480c6a30 0x7f05480d08f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.134+0000 7f054de00640 1 --2- 172.21.15.16:0/1393137947 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548159060 0x7f05480c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.134+0000 7f054de00640 1 -- 172.21.15.16:0/1393137947 >> 172.21.15.16:0/1393137947 conn(0x7f05480cfde0 msgr2=0x7f05480d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.135+0000 7f054de00640 1 -- 172.21.15.16:0/1393137947 shutdown_connections 2024-08-17T02:45:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.135+0000 7f054de00640 1 -- 172.21.15.16:0/1393137947 wait complete. 2024-08-17T02:45:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.135+0000 7ffa40a00640 1 -- 172.21.15.16:0/265474804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20005130 msgr2=0x7ffa2000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.135+0000 7ffa40a00640 1 --2- 172.21.15.16:0/265474804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20005130 0x7ffa2000b9d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffa3400c410 tx=0x7ffa34007940 comp rx=0 tx=0).stop 2024-08-17T02:45:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.135+0000 7ffa40a00640 1 -- 172.21.15.16:0/265474804 shutdown_connections 2024-08-17T02:45:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.135+0000 7ffa40a00640 1 --2- 172.21.15.16:0/265474804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20005130 0x7ffa2000b9d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.135+0000 7ffa40a00640 1 -- 172.21.15.16:0/265474804 >> 172.21.15.16:0/265474804 conn(0x7ffa3c098e30 msgr2=0x7ffa3c06eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.136+0000 7efdae200640 1 --2- 172.21.15.16:0/1799905923 >> v2:172.21.15.16:3300/0 conn(0x7efda80b1270 0x7efda80abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.136+0000 7f054de00640 1 Processor -- start 2024-08-17T02:45:26.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.136+0000 7f054de00640 1 -- start start 2024-08-17T02:45:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7ffa40a00640 1 -- 172.21.15.16:0/265474804 shutdown_connections 2024-08-17T02:45:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.136+0000 7fed9fe00640 1 -- 172.21.15.16:0/2372119660 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7feda403ca20 con 0x7fedac15e140 2024-08-17T02:45:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7f054de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05480c6a30 0x7f05480b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7f054de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0548159060 0x7f05480b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.136+0000 7fed9fe00640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fed8c065150 0x7fed8c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7fed9fe00640 1 -- 172.21.15.16:0/2372119660 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feda40a4230 con 0x7fedac15e140 2024-08-17T02:45:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7fed9fe00640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] conn(0x7fed8c06d360 0x7fed8c06f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7fed9fe00640 1 -- 172.21.15.16:0/2372119660 --> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed8c06fe20 con 0x7fed8c06d360 2024-08-17T02:45:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7fed9fe00640 1 -- 172.21.15.16:0/2372119660 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7feda40a4520 con 0x7fedac15e140 2024-08-17T02:45:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.138+0000 7ffa40a00640 1 -- 172.21.15.16:0/265474804 wait complete. 2024-08-17T02:45:26.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7f054de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548159430 0x7f05480b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7f054de00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05481578e0 con 0x7f05480c6a30 2024-08-17T02:45:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7f054de00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548157bc0 con 0x7f0548159060 2024-08-17T02:45:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7f054de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0548157ea0 con 0x7f0548159430 2024-08-17T02:45:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.138+0000 7fedabe00640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] conn(0x7fed8c06d360 0x7fed8c06f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.138+0000 7f0546a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05480c6a30 0x7f05480b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.138+0000 7f0546a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05480c6a30 0x7f05480b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57936/0 (socket says 172.21.15.16:57936) 2024-08-17T02:45:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.138+0000 7f0546a00640 1 -- 172.21.15.16:0/3852266727 learned_addr learned my addr 172.21.15.16:0/3852266727 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.138+0000 7f5f03400640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] conn(0x7f5ee406d4b0 0x7f5ee406f8c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.138+0000 7f0546000640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0548159060 0x7f05480b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.138+0000 7f0547400640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548159430 0x7f05480b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.136+0000 7efdace00640 1 --2- 172.21.15.16:0/1799905923 >> v2:172.21.15.16:3300/0 conn(0x7efda80b1270 0x7efda80abc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.136+0000 7efdace00640 1 -- 172.21.15.16:0/1799905923 >> v1:172.21.15.16:6789/0 conn(0x7efda81403f0 legacy=0x7efda80ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7f1a43000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a3c0b4020 0x7f1a3c09d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.136+0000 7efdace00640 1 -- 172.21.15.16:0/1799905923 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda80cace0 con 0x7efda80c9200 2024-08-17T02:45:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7f1a43000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a3c0c6250 0x7f1a3c157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7f1a43000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a3c15d610 0x7f1a3c158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f0547400640 1 -- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0548159060 msgr2=0x7f05480b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f0547400640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0548159060 0x7f05480b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f0547400640 1 -- 172.21.15.16:0/3852266727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05480c6a30 msgr2=0x7f05480b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f0547400640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05480c6a30 0x7f05480b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f0547400640 1 -- 172.21.15.16:0/3852266727 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0538009390 con 0x7f0548159430 2024-08-17T02:45:26.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f0546a00640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05480c6a30 0x7f05480b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:26.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7fedabe00640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] conn(0x7fed8c06d360 0x7fed8c06f770 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7fedb0a00640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fed8c065150 0x7fed8c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7efdace00640 1 --2- 172.21.15.16:0/1799905923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda80c9200 0x7efda80b0d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efda806bf10 tx=0x7efd940090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=831414be2a6d8aeb server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7f1a43000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c176bf0 con 0x7f1a3c0c6250 2024-08-17T02:45:26.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7f1a43000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c177490 con 0x7f1a3c0b4020 2024-08-17T02:45:26.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.137+0000 7f1a43000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c176870 con 0x7f1a3c15d610 2024-08-17T02:45:26.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.138+0000 7f1a41000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a3c15d610 0x7f1a3c158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f5efa000640 1 -- 172.21.15.16:0/3707657471 <== osd.1 v2:172.21.15.16:6800/37179 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5ee406ff70 con 0x7f5ee406d4b0 2024-08-17T02:45:26.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.138+0000 7f1a41000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a3c15d610 0x7f1a3c158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:52994/0 (socket says 172.21.15.16:52994) 2024-08-17T02:45:26.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.138+0000 7f1a41000640 1 -- 172.21.15.16:0/1077561346 learned_addr learned my addr 172.21.15.16:0/1077561346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f1a41000640 1 -- 172.21.15.16:0/1077561346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a3c0b4020 msgr2=0x7f1a3c09d340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-17T02:45:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f1a41000640 1 --2- 172.21.15.16:0/1077561346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a3c0b4020 0x7f1a3c09d340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f1a41000640 1 -- 172.21.15.16:0/1077561346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a3c0c6250 msgr2=0x7f1a3c157c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-17T02:45:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f1a41000640 1 --2- 172.21.15.16:0/1077561346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a3c0c6250 0x7f1a3c157c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f1a41000640 1 -- 172.21.15.16:0/1077561346 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24009390 con 0x7f1a3c15d610 2024-08-17T02:45:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f0547400640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548159430 0x7f05480b4090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f054000c7e0 tx=0x7f05400025b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7efda7e00640 1 -- 172.21.15.16:0/1799905923 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9402ae80 con 0x7efda80c9200 2024-08-17T02:45:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7efda7e00640 1 -- 172.21.15.16:0/1799905923 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd94032440 con 0x7efda80c9200 2024-08-17T02:45:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7efda7e00640 1 -- 172.21.15.16:0/1799905923 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd94032800 con 0x7efda80c9200 2024-08-17T02:45:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.140+0000 7f1a41000640 1 --2- 172.21.15.16:0/1077561346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a3c15d610 0x7f1a3c158160 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009850 tx=0x7f1a24032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.140+0000 7f1a33400640 1 -- 172.21.15.16:0/1077561346 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2402be20 con 0x7f1a3c15d610 2024-08-17T02:45:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.140+0000 7f1a33400640 1 -- 172.21.15.16:0/1077561346 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a24031a60 con 0x7f1a3c15d610 2024-08-17T02:45:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.141+0000 7f1a43000640 1 -- 172.21.15.16:0/1077561346 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c176e40 con 0x7f1a3c15d610 2024-08-17T02:45:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.141+0000 7f1a33400640 1 -- 172.21.15.16:0/1077561346 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24031e60 con 0x7f1a3c15d610 2024-08-17T02:45:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.141+0000 7f1a43000640 1 -- 172.21.15.16:0/1077561346 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c09d880 con 0x7f1a3c15d610 2024-08-17T02:45:26.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.140+0000 7fedb0a00640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fed8c065150 0x7fed8c067600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feda00011e0 tx=0x7feda0007c80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.139+0000 7f053e000640 1 -- 172.21.15.16:0/3852266727 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054000a640 con 0x7f0548159430 2024-08-17T02:45:26.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.140+0000 7f053e000640 1 -- 172.21.15.16:0/3852266727 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054000a7a0 con 0x7f0548159430 2024-08-17T02:45:26.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.140+0000 7f053e000640 1 -- 172.21.15.16:0/3852266727 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054000ab80 con 0x7f0548159430 2024-08-17T02:45:26.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.140+0000 7f054de00640 1 -- 172.21.15.16:0/3852266727 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0534009450 con 0x7f0548159430 2024-08-17T02:45:26.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.140+0000 7f054de00640 1 -- 172.21.15.16:0/3852266727 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0548158460 con 0x7f0548159430 2024-08-17T02:45:26.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.140+0000 7efdb0200640 1 -- 172.21.15.16:0/1799905923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda80c9200 msgr2=0x7efda80b0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.140+0000 7efdb0200640 1 --2- 172.21.15.16:0/1799905923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda80c9200 0x7efda80b0d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efda806bf10 tx=0x7efd940090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.141+0000 7fed9fe00640 1 -- 172.21.15.16:0/2372119660 <== osd.4 v2:172.21.15.142:6818/37611 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fed8c06fe20 con 0x7fed8c06d360 2024-08-17T02:45:26.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.143+0000 7f053e000640 1 -- 172.21.15.16:0/3852266727 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f0540012420 con 0x7f0548159430 2024-08-17T02:45:26.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.143+0000 7f054de00640 1 -- 172.21.15.16:0/3852266727 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0548063d20 con 0x7f0548159430 2024-08-17T02:45:26.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.141+0000 7ffa40a00640 1 Processor -- start 2024-08-17T02:45:26.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.142+0000 7ffa40a00640 1 -- start start 2024-08-17T02:45:26.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.143+0000 7f053e000640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0528065220 0x7f05280676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.143+0000 7f053e000640 1 -- 172.21.15.16:0/3852266727 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0540081f00 con 0x7f0548159430 2024-08-17T02:45:26.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.144+0000 7f053e000640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] conn(0x7f052806d430 0x7f052806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.144+0000 7f053e000640 1 -- 172.21.15.16:0/3852266727 --> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f052806fef0 con 0x7f052806d430 2024-08-17T02:45:26.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.144+0000 7f1a33400640 1 -- 172.21.15.16:0/1077561346 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1a24039050 con 0x7f1a3c15d610 2024-08-17T02:45:26.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.144+0000 7f053e000640 1 -- 172.21.15.16:0/3852266727 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0540084540 con 0x7f0548159430 2024-08-17T02:45:26.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.144+0000 7f0546a00640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0528065220 0x7f05280676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.144+0000 7f0546000640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] conn(0x7f052806d430 0x7f052806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.145+0000 7ffa40a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20005130 0x7ffa3c0b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.145+0000 7ffa40a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3c157be0 0x7ffa3c0b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.145+0000 7f0546a00640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0528065220 0x7f05280676d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0538011640 tx=0x7f05380090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.145+0000 7f1a43000640 1 -- 172.21.15.16:0/1077561346 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a3c063d20 con 0x7f1a3c15d610 2024-08-17T02:45:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.145+0000 7ffa40a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa3c0c8290 0x7ffa3c0c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7efdb0200640 1 -- 172.21.15.16:0/1799905923 shutdown_connections 2024-08-17T02:45:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7efdb0200640 1 --2- 172.21.15.16:0/1799905923 >> v2:172.21.15.16:3300/0 conn(0x7efda80b1270 0x7efda80abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7efdb0200640 1 --2- 172.21.15.16:0/1799905923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda80c9200 0x7efda80b0d30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7efdb0200640 1 -- 172.21.15.16:0/1799905923 >> 172.21.15.16:0/1799905923 conn(0x7efda80cc480 msgr2=0x7efda80687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7f1a33400640 1 --2- 172.21.15.16:0/1077561346 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1a18065220 0x7f1a180676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7f1a33400640 1 -- 172.21.15.16:0/1077561346 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a240a2dd0 con 0x7f1a3c15d610 2024-08-17T02:45:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7ffa40a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c162dd0 con 0x7ffa20005130 2024-08-17T02:45:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7ffa40a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c0b24b0 con 0x7ffa3c0c8290 2024-08-17T02:45:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7ffa3b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20005130 0x7ffa3c0b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7ffa40a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3c0c9060 con 0x7ffa3c157be0 2024-08-17T02:45:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7ffa32a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3c157be0 0x7ffa3c0b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7f1a33400640 1 --2- 172.21.15.16:0/1077561346 >> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] conn(0x7f1a1806d430 0x7f1a1806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7f1a33400640 1 -- 172.21.15.16:0/1077561346 --> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a1806fef0 con 0x7f1a1806d430 2024-08-17T02:45:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7f1a33400640 1 -- 172.21.15.16:0/1077561346 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a240a6020 con 0x7f1a3c15d610 2024-08-17T02:45:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.147+0000 7f1a3b400640 1 --2- 172.21.15.16:0/1077561346 >> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] conn(0x7f1a1806d430 0x7f1a1806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.147+0000 7f1a3be00640 1 --2- 172.21.15.16:0/1077561346 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1a18065220 0x7f1a180676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7ffa3be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa3c0c8290 0x7ffa3c0c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7ffa32a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3c157be0 0x7ffa3c0b1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:53022/0 (socket says 172.21.15.16:53022) 2024-08-17T02:45:26.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.146+0000 7ffa32a00640 1 -- 172.21.15.16:0/2791455264 learned_addr learned my addr 172.21.15.16:0/2791455264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.147+0000 7ffa32a00640 1 -- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa3c0c8290 msgr2=0x7ffa3c0c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.147+0000 7f0546000640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] conn(0x7f052806d430 0x7f052806f840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.147+0000 7ffa32a00640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa3c0c8290 0x7ffa3c0c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.147+0000 7ffa32a00640 1 -- 172.21.15.16:0/2791455264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20005130 msgr2=0x7ffa3c0b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.147+0000 7ffa32a00640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20005130 0x7ffa3c0b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.147+0000 7ffa32a00640 1 -- 172.21.15.16:0/2791455264 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c0c9340 con 0x7ffa3c157be0 2024-08-17T02:45:26.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.147+0000 7ffa3be00640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa3c0c8290 0x7ffa3c0c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:26.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.147+0000 7ffa3b400640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa20005130 0x7ffa3c0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:26.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.148+0000 7f1a3be00640 1 --2- 172.21.15.16:0/1077561346 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1a18065220 0x7f1a180676d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c158e80 tx=0x7f1a3400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.148+0000 7ffa32a00640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3c157be0 0x7ffa3c0b1f70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400ea50 tx=0x7ffa240090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.148+0000 7f053e000640 1 -- 172.21.15.16:0/3852266727 <== osd.6 v2:172.21.15.142:6808/37610 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f052806fef0 con 0x7f052806d430 2024-08-17T02:45:26.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.148+0000 7efdb0200640 1 -- 172.21.15.16:0/1799905923 shutdown_connections 2024-08-17T02:45:26.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.148+0000 7fc1a5000640 1 -- 172.21.15.16:0/1367844359 --> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc19800ea60 con 0x7fc18006d2c0 2024-08-17T02:45:26.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.149+0000 7fc195600640 1 -- 172.21.15.16:0/1367844359 <== osd.0 v2:172.21.15.16:6824/37182 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc19800ea60 con 0x7fc18006d2c0 2024-08-17T02:45:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.149+0000 7fc1a5000640 1 -- 172.21.15.16:0/1367844359 >> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] conn(0x7fc18006d2c0 msgr2=0x7fc18006f6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.149+0000 7fc1a5000640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] conn(0x7fc18006d2c0 0x7fc18006f6d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.148+0000 7ffa33400640 1 -- 172.21.15.16:0/2791455264 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2400c940 con 0x7ffa3c157be0 2024-08-17T02:45:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.148+0000 7ffa33400640 1 -- 172.21.15.16:0/2791455264 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2400caa0 con 0x7ffa3c157be0 2024-08-17T02:45:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.148+0000 7ffa40a00640 1 -- 172.21.15.16:0/2791455264 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3c0c9620 con 0x7ffa3c157be0 2024-08-17T02:45:26.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.148+0000 7ffa33400640 1 -- 172.21.15.16:0/2791455264 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2400ce60 con 0x7ffa3c157be0 2024-08-17T02:45:26.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.148+0000 7ffa40a00640 1 -- 172.21.15.16:0/2791455264 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3c0b37d0 con 0x7ffa3c157be0 2024-08-17T02:45:26.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.148+0000 7efdb0200640 1 -- 172.21.15.16:0/1799905923 wait complete. 2024-08-17T02:45:26.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.150+0000 7efdb0200640 1 Processor -- start 2024-08-17T02:45:26.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.150+0000 7efdb0200640 1 -- start start 2024-08-17T02:45:26.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.148+0000 7f1a3b400640 1 --2- 172.21.15.16:0/1077561346 >> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] conn(0x7f1a1806d430 0x7f1a1806f840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.151+0000 7ffa40a00640 1 -- 172.21.15.16:0/2791455264 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa3c071050 con 0x7ffa3c157be0 2024-08-17T02:45:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.151+0000 7fc1a5000640 1 -- 172.21.15.16:0/1367844359 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc180065220 msgr2=0x7fc1800676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.151+0000 7fc1a5000640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc180065220 0x7fc1800676d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc190007c60 tx=0x7fc190008040 comp rx=0 tx=0).stop 2024-08-17T02:45:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.151+0000 7fc1a5000640 1 -- 172.21.15.16:0/1367844359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1980e0140 msgr2=0x7fc1980cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.151+0000 7fc1a5000640 1 --2- 172.21.15.16:0/1367844359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc1980e0140 0x7fc1980cfb60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc19800f250 tx=0x7fc18c032000 comp rx=0 tx=0).stop 2024-08-17T02:45:26.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.151+0000 7fc19e800640 1 -- 172.21.15.16:0/1367844359 reap_dead start 2024-08-17T02:45:26.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.151+0000 7f1a33400640 1 -- 172.21.15.16:0/1077561346 <== osd.3 v2:172.21.15.16:6808/37181 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1a1806fef0 con 0x7f1a1806d430 2024-08-17T02:45:26.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.151+0000 7fc1a5000640 1 -- 172.21.15.16:0/1367844359 shutdown_connections 2024-08-17T02:45:26.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.151+0000 7fc1a5000640 1 -- 172.21.15.16:0/1367844359 >> 172.21.15.16:0/1367844359 conn(0x7fc198057870 msgr2=0x7fc1980d98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7fc1a5000640 1 -- 172.21.15.16:0/1367844359 shutdown_connections 2024-08-17T02:45:26.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7fc1a5000640 1 -- 172.21.15.16:0/1367844359 wait complete. 2024-08-17T02:45:26.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7ffa33400640 1 -- 172.21.15.16:0/2791455264 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7ffa24013680 con 0x7ffa3c157be0 2024-08-17T02:45:26.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7efdb0200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efda80b1270 0x7efda8158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7efdb0200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda80c9200 0x7efda81387a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7efdb0200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda81403f0 0x7efda80af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7efdb0200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda8157420 con 0x7efda80b1270 2024-08-17T02:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7efdb0200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda8136c40 con 0x7efda81403f0 2024-08-17T02:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7efdb0200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda8159210 con 0x7efda80c9200 2024-08-17T02:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7efdad800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efda80b1270 0x7efda8158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7efdace00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda80c9200 0x7efda81387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7efdace00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda80c9200 0x7efda81387a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:53024/0 (socket says 172.21.15.16:53024) 2024-08-17T02:45:26.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.152+0000 7efdace00640 1 -- 172.21.15.16:0/726001866 learned_addr learned my addr 172.21.15.16:0/726001866 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.153+0000 7efdae200640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda81403f0 0x7efda80af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.153+0000 7ffa33400640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ffa18065220 0x7ffa180676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.153+0000 7ffa33400640 1 -- 172.21.15.16:0/2791455264 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffa240825e0 con 0x7ffa3c157be0 2024-08-17T02:45:26.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.154+0000 7ffa33400640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] conn(0x7ffa1806d430 0x7ffa1806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.154+0000 7ffa33400640 1 -- 172.21.15.16:0/2791455264 --> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa1806fef0 con 0x7ffa1806d430 2024-08-17T02:45:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.154+0000 7ffa33400640 1 -- 172.21.15.16:0/2791455264 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa24084b40 con 0x7ffa3c157be0 2024-08-17T02:45:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.153+0000 7efdae200640 1 -- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda80c9200 msgr2=0x7efda81387a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.153+0000 7efdae200640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efda80c9200 0x7efda81387a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.153+0000 7efdae200640 1 -- 172.21.15.16:0/726001866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efda80b1270 msgr2=0x7efda8158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.155+0000 7efdae200640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efda80b1270 0x7efda8158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.155+0000 7efdae200640 1 -- 172.21.15.16:0/726001866 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd94009390 con 0x7efda81403f0 2024-08-17T02:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.155+0000 7efdad800640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efda80b1270 0x7efda8158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.155+0000 7efdae200640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda81403f0 0x7efda80af3d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efda000c650 tx=0x7efda0008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.155+0000 7efda4c00640 1 -- 172.21.15.16:0/726001866 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda0011600 con 0x7efda81403f0 2024-08-17T02:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.156+0000 7efda4c00640 1 -- 172.21.15.16:0/726001866 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda0011760 con 0x7efda81403f0 2024-08-17T02:45:26.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.156+0000 7ffa3b400640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ffa18065220 0x7ffa180676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.156+0000 7efda4c00640 1 -- 172.21.15.16:0/726001866 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda0011b00 con 0x7efda81403f0 2024-08-17T02:45:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.156+0000 7efdb0200640 1 -- 172.21.15.16:0/726001866 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda81594f0 con 0x7efda81403f0 2024-08-17T02:45:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.156+0000 7efdb0200640 1 -- 172.21.15.16:0/726001866 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda8137080 con 0x7efda81403f0 2024-08-17T02:45:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.157+0000 7ffa3be00640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] conn(0x7ffa1806d430 0x7ffa1806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.157+0000 7ffa3b400640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ffa18065220 0x7ffa180676d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0cf330 tx=0x7ffa2c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.158+0000 7ffa3be00640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] conn(0x7ffa1806d430 0x7ffa1806f840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.159+0000 7ffa33400640 1 -- 172.21.15.16:0/2791455264 <== osd.5 v2:172.21.15.142:6800/37609 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ffa1806fef0 con 0x7ffa1806d430 2024-08-17T02:45:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.157+0000 7efdb0200640 1 -- 172.21.15.16:0/726001866 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efda8064db0 con 0x7efda81403f0 2024-08-17T02:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.160+0000 7efda4c00640 1 -- 172.21.15.16:0/726001866 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7efda0021350 con 0x7efda81403f0 2024-08-17T02:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.161+0000 7efda4c00640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efd88065220 0x7efd880676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.161+0000 7efda4c00640 1 -- 172.21.15.16:0/726001866 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efda0081fb0 con 0x7efda81403f0 2024-08-17T02:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.162+0000 7efdad800640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efd88065220 0x7efd880676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.162+0000 7efda4c00640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] conn(0x7efd8806d430 0x7efd8806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.162+0000 7efda4c00640 1 -- 172.21.15.16:0/726001866 --> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd8806fef0 con 0x7efd8806d430 2024-08-17T02:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.162+0000 7efda4c00640 1 -- 172.21.15.16:0/726001866 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7efda00844b0 con 0x7efda81403f0 2024-08-17T02:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.162+0000 7efdace00640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] conn(0x7efd8806d430 0x7efd8806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.162+0000 7efdad800640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efd88065220 0x7efd880676d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efd9c002230 tx=0x7efd9c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.165+0000 7efdace00640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] conn(0x7efd8806d430 0x7efd8806f840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.166+0000 7efda4c00640 1 -- 172.21.15.16:0/726001866 <== osd.7 v2:172.21.15.142:6816/37634 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7efd8806fef0 con 0x7efd8806d430 2024-08-17T02:45:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.167+0000 7f4ba7e00640 1 -- 172.21.15.16:0/1972952161 --> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4ba80aa520 con 0x7f4b8806d4b0 2024-08-17T02:45:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.168+0000 7f4b9e000640 1 -- 172.21.15.16:0/1972952161 <== osd.2 v2:172.21.15.16:6816/37183 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4ba80aa520 con 0x7f4b8806d4b0 2024-08-17T02:45:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.168+0000 7f4ba7e00640 1 -- 172.21.15.16:0/1972952161 >> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] conn(0x7f4b8806d4b0 msgr2=0x7f4b8806f8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.168+0000 7f4ba7e00640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] conn(0x7f4b8806d4b0 0x7f4b8806f8c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.169+0000 7f4ba7e00640 1 -- 172.21.15.16:0/1972952161 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4b880652a0 msgr2=0x7f4b88067750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.169+0000 7f4ba7e00640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4b880652a0 0x7f4b88067750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80cc4d0 tx=0x7f4b940090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.169+0000 7f4ba7e00640 1 -- 172.21.15.16:0/1972952161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba806e570 msgr2=0x7f4ba8144c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.169+0000 7f4ba7e00640 1 --2- 172.21.15.16:0/1972952161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ba806e570 0x7f4ba8144c40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000ed60 tx=0x7f4ba000b040 comp rx=0 tx=0).stop 2024-08-17T02:45:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.169+0000 7f4ba7400640 1 -- 172.21.15.16:0/1972952161 reap_dead start 2024-08-17T02:45:26.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.170+0000 7f4ba7e00640 1 -- 172.21.15.16:0/1972952161 shutdown_connections 2024-08-17T02:45:26.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.170+0000 7f4ba7e00640 1 -- 172.21.15.16:0/1972952161 >> 172.21.15.16:0/1972952161 conn(0x7f4ba80cb790 msgr2=0x7f4ba8145cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.170+0000 7f1a43000640 1 -- 172.21.15.16:0/1077561346 --> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1a3c063d20 con 0x7f1a1806d430 2024-08-17T02:45:26.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.170+0000 7f4ba7e00640 1 -- 172.21.15.16:0/1972952161 shutdown_connections 2024-08-17T02:45:26.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.170+0000 7f4ba7e00640 1 -- 172.21.15.16:0/1972952161 wait complete. 2024-08-17T02:45:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.172+0000 7f5f09a00640 1 -- 172.21.15.16:0/3707657471 --> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5efc00d490 con 0x7f5ee406d4b0 2024-08-17T02:45:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.172+0000 7f5efa000640 1 -- 172.21.15.16:0/3707657471 <== osd.1 v2:172.21.15.16:6800/37179 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5efc00d490 con 0x7f5ee406d4b0 2024-08-17T02:45:26.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.172+0000 7f1a33400640 1 -- 172.21.15.16:0/1077561346 <== osd.3 v2:172.21.15.16:6808/37181 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1a3c063d20 con 0x7f1a1806d430 2024-08-17T02:45:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.174+0000 7f5f09a00640 1 -- 172.21.15.16:0/3707657471 >> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] conn(0x7f5ee406d4b0 msgr2=0x7f5ee406f8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.174+0000 7f5f09a00640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] conn(0x7f5ee406d4b0 0x7f5ee406f8c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.174+0000 7f5f09a00640 1 -- 172.21.15.16:0/3707657471 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ee40652a0 msgr2=0x7f5ee4067750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.174+0000 7f5f09a00640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ee40652a0 0x7f5ee4067750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4007a10 tx=0x7f5ef4001540 comp rx=0 tx=0).stop 2024-08-17T02:45:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.174+0000 7f5f09a00640 1 -- 172.21.15.16:0/3707657471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5efc0dfd60 msgr2=0x7f5efc038520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.174+0000 7f5f09a00640 1 --2- 172.21.15.16:0/3707657471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5efc0dfd60 0x7f5efc038520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000ec80 tx=0x7f5ef00090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.174+0000 7f5f03400640 1 -- 172.21.15.16:0/3707657471 reap_dead start 2024-08-17T02:45:26.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.174+0000 7f5f09a00640 1 -- 172.21.15.16:0/3707657471 shutdown_connections 2024-08-17T02:45:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.174+0000 7f5f09a00640 1 -- 172.21.15.16:0/3707657471 >> 172.21.15.16:0/3707657471 conn(0x7f5efc00a830 msgr2=0x7f5efc055e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 --> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fedac06cc70 con 0x7fed8c06d360 2024-08-17T02:45:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7f5f09a00640 1 -- 172.21.15.16:0/3707657471 shutdown_connections 2024-08-17T02:45:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7f5f09a00640 1 -- 172.21.15.16:0/3707657471 wait complete. 2024-08-17T02:45:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7fed9fe00640 1 -- 172.21.15.16:0/2372119660 <== osd.4 v2:172.21.15.142:6818/37611 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fedac06cc70 con 0x7fed8c06d360 2024-08-17T02:45:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7f1a43000640 1 -- 172.21.15.16:0/1077561346 >> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] conn(0x7f1a1806d430 msgr2=0x7f1a1806f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7f1a43000640 1 --2- 172.21.15.16:0/1077561346 >> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] conn(0x7f1a1806d430 0x7f1a1806f840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7f1a43000640 1 -- 172.21.15.16:0/1077561346 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1a18065220 msgr2=0x7f1a180676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7f1a43000640 1 --2- 172.21.15.16:0/1077561346 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1a18065220 0x7f1a180676d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c158e80 tx=0x7f1a3400b040 comp rx=0 tx=0).stop 2024-08-17T02:45:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7f1a43000640 1 -- 172.21.15.16:0/1077561346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a3c15d610 msgr2=0x7f1a3c158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7f1a43000640 1 --2- 172.21.15.16:0/1077561346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a3c15d610 0x7f1a3c158160 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009850 tx=0x7f1a24032000 comp rx=0 tx=0).stop 2024-08-17T02:45:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7f1a41000640 1 -- 172.21.15.16:0/1077561346 reap_dead start 2024-08-17T02:45:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7f1a43000640 1 -- 172.21.15.16:0/1077561346 shutdown_connections 2024-08-17T02:45:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7f1a43000640 1 -- 172.21.15.16:0/1077561346 >> 172.21.15.16:0/1077561346 conn(0x7f1a3c0cfde0 msgr2=0x7f1a3c06e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.175+0000 7f1a43000640 1 -- 172.21.15.16:0/1077561346 shutdown_connections 2024-08-17T02:45:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.176+0000 7f1a43000640 1 -- 172.21.15.16:0/1077561346 wait complete. 2024-08-17T02:45:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.179+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] conn(0x7fed8c06d360 msgr2=0x7fed8c06f770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.179+0000 7fedb3400640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] conn(0x7fed8c06d360 0x7fed8c06f770 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.230 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2024-08-17T02:45:26.230 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-17T02:45:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.180+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fed8c065150 msgr2=0x7fed8c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.180+0000 7fedb3400640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fed8c065150 0x7fed8c067600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feda00011e0 tx=0x7feda0007c80 comp rx=0 tx=0).stop 2024-08-17T02:45:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.180+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac15e140 msgr2=0x7fedac155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.180+0000 7fedb3400640 1 --2- 172.21.15.16:0/2372119660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedac15e140 0x7fedac155f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feda40099f0 tx=0x7feda4037000 comp rx=0 tx=0).stop 2024-08-17T02:45:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.181+0000 7fedb1400640 1 -- 172.21.15.16:0/2372119660 reap_dead start 2024-08-17T02:45:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.181+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 shutdown_connections 2024-08-17T02:45:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.181+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 >> 172.21.15.16:0/2372119660 conn(0x7fedac0cb9b0 msgr2=0x7fedac1567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.182+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 shutdown_connections 2024-08-17T02:45:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.182+0000 7fedb3400640 1 -- 172.21.15.16:0/2372119660 wait complete. 2024-08-17T02:45:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.184+0000 7f054de00640 1 -- 172.21.15.16:0/3852266727 --> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0548067150 con 0x7f052806d430 2024-08-17T02:45:26.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.185+0000 7f053e000640 1 -- 172.21.15.16:0/3852266727 <== osd.6 v2:172.21.15.142:6808/37610 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0548067150 con 0x7f052806d430 2024-08-17T02:45:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.185+0000 7f054de00640 1 -- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] conn(0x7f052806d430 msgr2=0x7f052806f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.185+0000 7f054de00640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] conn(0x7f052806d430 0x7f052806f840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.186+0000 7f054de00640 1 -- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0528065220 msgr2=0x7f05280676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.186+0000 7f054de00640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0528065220 0x7f05280676d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0538011640 tx=0x7f05380090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.186+0000 7f054de00640 1 -- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548159430 msgr2=0x7f05480b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.186+0000 7f054de00640 1 --2- 172.21.15.16:0/3852266727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548159430 0x7f05480b4090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f054000c7e0 tx=0x7f05400025b0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.186+0000 7f0547400640 1 -- 172.21.15.16:0/3852266727 reap_dead start 2024-08-17T02:45:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.186+0000 7f054de00640 1 -- 172.21.15.16:0/3852266727 shutdown_connections 2024-08-17T02:45:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.186+0000 7f054de00640 1 -- 172.21.15.16:0/3852266727 >> 172.21.15.16:0/3852266727 conn(0x7f05480cfde0 msgr2=0x7f05480c9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.187+0000 7f054de00640 1 -- 172.21.15.16:0/3852266727 shutdown_connections 2024-08-17T02:45:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.187+0000 7f054de00640 1 -- 172.21.15.16:0/3852266727 wait complete. 2024-08-17T02:45:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.188+0000 7ffa40a00640 1 -- 172.21.15.16:0/2791455264 --> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa3c0cbaf0 con 0x7ffa1806d430 2024-08-17T02:45:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.188+0000 7ffa33400640 1 -- 172.21.15.16:0/2791455264 <== osd.5 v2:172.21.15.142:6800/37609 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffa3c0cbaf0 con 0x7ffa1806d430 2024-08-17T02:45:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.189+0000 7ffa40a00640 1 -- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] conn(0x7ffa1806d430 msgr2=0x7ffa1806f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.189+0000 7ffa40a00640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] conn(0x7ffa1806d430 0x7ffa1806f840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.190+0000 7ffa40a00640 1 -- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ffa18065220 msgr2=0x7ffa180676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.190+0000 7ffa40a00640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ffa18065220 0x7ffa180676d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0cf330 tx=0x7ffa2c00b040 comp rx=0 tx=0).stop 2024-08-17T02:45:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.190+0000 7ffa40a00640 1 -- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3c157be0 msgr2=0x7ffa3c0b1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.190+0000 7ffa40a00640 1 --2- 172.21.15.16:0/2791455264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ffa3c157be0 0x7ffa3c0b1f70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400ea50 tx=0x7ffa240090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.190+0000 7ffa3be00640 1 -- 172.21.15.16:0/2791455264 reap_dead start 2024-08-17T02:45:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.190+0000 7ffa40a00640 1 -- 172.21.15.16:0/2791455264 shutdown_connections 2024-08-17T02:45:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.190+0000 7ffa40a00640 1 -- 172.21.15.16:0/2791455264 >> 172.21.15.16:0/2791455264 conn(0x7ffa3c098e30 msgr2=0x7ffa3c0ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.191+0000 7efdb0200640 1 -- 172.21.15.16:0/726001866 --> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efda8070740 con 0x7efd8806d430 2024-08-17T02:45:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.191+0000 7ffa40a00640 1 -- 172.21.15.16:0/2791455264 shutdown_connections 2024-08-17T02:45:26.237 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.191+0000 7ffa40a00640 1 -- 172.21.15.16:0/2791455264 wait complete. 2024-08-17T02:45:26.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-17T02:45:26.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.192+0000 7efda4c00640 1 -- 172.21.15.16:0/726001866 <== osd.7 v2:172.21.15.142:6816/37634 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efda8070740 con 0x7efd8806d430 2024-08-17T02:45:26.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.193+0000 7efdb0200640 1 -- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] conn(0x7efd8806d430 msgr2=0x7efd8806f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.193+0000 7efdb0200640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] conn(0x7efd8806d430 0x7efd8806f840 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.193+0000 7efdb0200640 1 -- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efd88065220 msgr2=0x7efd880676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.193+0000 7efdb0200640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efd88065220 0x7efd880676d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efd9c002230 tx=0x7efd9c009040 comp rx=0 tx=0).stop 2024-08-17T02:45:26.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.193+0000 7efdb0200640 1 -- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda81403f0 msgr2=0x7efda80af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.193+0000 7efdb0200640 1 --2- 172.21.15.16:0/726001866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efda81403f0 0x7efda80af3d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efda000c650 tx=0x7efda0008040 comp rx=0 tx=0).stop 2024-08-17T02:45:26.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.193+0000 7efdae200640 1 -- 172.21.15.16:0/726001866 reap_dead start 2024-08-17T02:45:26.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.193+0000 7efdb0200640 1 -- 172.21.15.16:0/726001866 shutdown_connections 2024-08-17T02:45:26.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.193+0000 7efdb0200640 1 -- 172.21.15.16:0/726001866 >> 172.21.15.16:0/726001866 conn(0x7efda80cc480 msgr2=0x7efda806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.193+0000 7efdb0200640 1 -- 172.21.15.16:0/726001866 shutdown_connections 2024-08-17T02:45:26.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.194+0000 7efdb0200640 1 -- 172.21.15.16:0/726001866 wait complete. 2024-08-17T02:45:26.240 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2024-08-17T02:45:26.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-17T02:45:26.241 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:26.241 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-17T02:45:26.242 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:26.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-17T02:45:26.243 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:26.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-17T02:45:26.244 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2024-08-17T02:45:26.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-17T02:45:26.245 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:26.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-17T02:45:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.543+0000 7f4396000640 1 Processor -- start 2024-08-17T02:45:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.544+0000 7f4396000640 1 -- start start 2024-08-17T02:45:26.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.546+0000 7f4396000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43880ddd90 0x7f43880de160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.546+0000 7f4396000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f438800c040 0x7f4388034160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.546+0000 7f4396000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4388053020 0x7f438804fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.546+0000 7f4396000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4388054130 con 0x7f4388053020 2024-08-17T02:45:26.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.546+0000 7f4396000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4388053520 con 0x7f438800c040 2024-08-17T02:45:26.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.546+0000 7f4396000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4388053800 con 0x7f43880ddd90 2024-08-17T02:45:26.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.546+0000 7f438fe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f438800c040 0x7f4388034160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.547+0000 7f4394c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43880ddd90 0x7f43880de160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.547+0000 7f4394c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43880ddd90 0x7f43880de160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:53032/0 (socket says 172.21.15.16:53032) 2024-08-17T02:45:26.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.547+0000 7f4394c00640 1 -- 172.21.15.16:0/1633048599 learned_addr learned my addr 172.21.15.16:0/1633048599 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.547+0000 7f4394c00640 1 -- 172.21.15.16:0/1633048599 >> v2:172.21.15.142:3300/0 conn(0x7f438800c040 msgr2=0x7f4388034160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.547+0000 7f4394c00640 1 --2- 172.21.15.16:0/1633048599 >> v2:172.21.15.142:3300/0 conn(0x7f438800c040 0x7f4388034160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.547+0000 7f4394c00640 1 -- 172.21.15.16:0/1633048599 >> v2:172.21.15.16:3300/0 conn(0x7f4388053020 msgr2=0x7f438804fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.547+0000 7f4394c00640 1 --2- 172.21.15.16:0/1633048599 >> v2:172.21.15.16:3300/0 conn(0x7f4388053020 0x7f438804fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.548+0000 7f4394c00640 1 -- 172.21.15.16:0/1633048599 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4388053ae0 con 0x7f43880ddd90 2024-08-17T02:45:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.549+0000 7f4394c00640 1 --2- 172.21.15.16:0/1633048599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43880ddd90 0x7f43880de160 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4388007a00 tx=0x7f43840091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fe0f77e75af0112 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.550+0000 7f4fa1200640 1 Processor -- start 2024-08-17T02:45:26.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.550+0000 7f4fa1200640 1 -- start start 2024-08-17T02:45:26.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.550+0000 7f438f400640 1 -- 172.21.15.16:0/1633048599 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438402ad10 con 0x7f43880ddd90 2024-08-17T02:45:26.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.550+0000 7f438f400640 1 -- 172.21.15.16:0/1633048599 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438402ae70 con 0x7f43880ddd90 2024-08-17T02:45:26.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.550+0000 7f438f400640 1 -- 172.21.15.16:0/1633048599 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4384032650 con 0x7f43880ddd90 2024-08-17T02:45:26.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.551+0000 7f4396000640 1 -- 172.21.15.16:0/1633048599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43880ddd90 msgr2=0x7f43880de160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.551+0000 7f4396000640 1 --2- 172.21.15.16:0/1633048599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43880ddd90 0x7f43880de160 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4388007a00 tx=0x7f43840091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.552+0000 7f4396000640 1 -- 172.21.15.16:0/1633048599 shutdown_connections 2024-08-17T02:45:26.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.552+0000 7f4396000640 1 --2- 172.21.15.16:0/1633048599 >> v2:172.21.15.16:3300/0 conn(0x7f4388053020 0x7f438804fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.552+0000 7f4396000640 1 --2- 172.21.15.16:0/1633048599 >> v2:172.21.15.142:3300/0 conn(0x7f438800c040 0x7f4388034160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.552+0000 7f4fa1200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4f940dc9b0 0x7f4f940e26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.552+0000 7f4fa1200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f940e2c10 0x7f4f94006a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.552+0000 7f4fa1200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f940e3650 con 0x7f4f940dc5e0 2024-08-17T02:45:26.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.552+0000 7f4fa1200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f94053f80 con 0x7f4f940e2c10 2024-08-17T02:45:26.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.552+0000 7f4fa1200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f940506f0 con 0x7f4f940dc9b0 2024-08-17T02:45:26.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.552+0000 7f4396000640 1 --2- 172.21.15.16:0/1633048599 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43880ddd90 0x7f43880de160 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.552+0000 7f4396000640 1 -- 172.21.15.16:0/1633048599 >> 172.21.15.16:0/1633048599 conn(0x7f4388009fa0 msgr2=0x7f438800a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.553+0000 7f4396000640 1 -- 172.21.15.16:0/1633048599 shutdown_connections 2024-08-17T02:45:26.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.552+0000 7f8a83200640 1 Processor -- start 2024-08-17T02:45:26.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.552+0000 7f8a83200640 1 -- start start 2024-08-17T02:45:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.553+0000 7f4f9be00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f940e2c10 0x7f4f94006a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.553+0000 7f4f9be00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f940e2c10 0x7f4f94006a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57984/0 (socket says 172.21.15.16:57984) 2024-08-17T02:45:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.553+0000 7f4f9be00640 1 -- 172.21.15.16:0/1067368118 learned_addr learned my addr 172.21.15.16:0/1067368118 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.553+0000 7f4396000640 1 -- 172.21.15.16:0/1633048599 wait complete. 2024-08-17T02:45:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.554+0000 7f4f9be00640 1 -- 172.21.15.16:0/1067368118 >> v2:172.21.15.142:3300/0 conn(0x7f4f940dc9b0 msgr2=0x7f4f940e26d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-17T02:45:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.554+0000 7f4f9be00640 1 --2- 172.21.15.16:0/1067368118 >> v2:172.21.15.142:3300/0 conn(0x7f4f940dc9b0 0x7f4f940e26d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.554+0000 7f4f9be00640 1 -- 172.21.15.16:0/1067368118 >> v1:172.21.15.142:6789/0 conn(0x7f4f940dc5e0 legacy=0x7f4f94053870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.554+0000 7f4f9be00640 1 -- 172.21.15.16:0/1067368118 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f940e3090 con 0x7f4f940e2c10 2024-08-17T02:45:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.555+0000 7f8a83200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a84158e20 0x7f8a840a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.555+0000 7f8a83200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a840ab520 con 0x7f8a841591f0 2024-08-17T02:45:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.555+0000 7f8a83200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a840aa110 con 0x7f8a840afed0 2024-08-17T02:45:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.555+0000 7f8a83200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a840aa3f0 con 0x7f8a84158e20 2024-08-17T02:45:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.556+0000 7f4396000640 1 Processor -- start 2024-08-17T02:45:26.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.556+0000 7f4f9be00640 1 --2- 172.21.15.16:0/1067368118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f940e2c10 0x7f4f94006a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4f94055500 tx=0x7f4f880091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4831f8fe70dd85b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.555+0000 7f8a81e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a84158e20 0x7f8a840a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.556+0000 7f8a81e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a84158e20 0x7f8a840a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57994/0 (socket says 172.21.15.16:57994) 2024-08-17T02:45:26.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.556+0000 7f8a81e00640 1 -- 172.21.15.16:0/293719413 learned_addr learned my addr 172.21.15.16:0/293719413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.556+0000 7f4396000640 1 -- start start 2024-08-17T02:45:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.556+0000 7f8a81400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8a841591f0 0x7f8a840af7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46140/0 (socket says 172.21.15.16:46140) 2024-08-17T02:45:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.556+0000 7f4f9a000640 1 -- 172.21.15.16:0/1067368118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88032440 con 0x7f4f940e2c10 2024-08-17T02:45:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.557+0000 7f4f9a000640 1 -- 172.21.15.16:0/1067368118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f880325a0 con 0x7f4f940e2c10 2024-08-17T02:45:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.557+0000 7f4f9a000640 1 -- 172.21.15.16:0/1067368118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88032980 con 0x7f4f940e2c10 2024-08-17T02:45:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.556+0000 7f8a82800640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f8a840afed0 0x7f8a84068920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:34964/0 (socket says 172.21.15.16:34964) 2024-08-17T02:45:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.557+0000 7fcd75200640 1 Processor -- start 2024-08-17T02:45:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.557+0000 7f754f400640 1 Processor -- start 2024-08-17T02:45:26.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.557+0000 7f8a81e00640 1 -- 172.21.15.16:0/293719413 >> v1:172.21.15.142:6789/0 conn(0x7f8a840afed0 legacy=0x7f8a84068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.557+0000 7f8a81e00640 1 -- 172.21.15.16:0/293719413 >> v1:172.21.15.16:6789/0 conn(0x7f8a841591f0 legacy=0x7f8a840af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.557+0000 7f8a81e00640 1 -- 172.21.15.16:0/293719413 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a840aa6d0 con 0x7f8a84158e20 2024-08-17T02:45:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.557+0000 7fcd75200640 1 -- start start 2024-08-17T02:45:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f4fa1200640 1 -- 172.21.15.16:0/1067368118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f940e2c10 msgr2=0x7f4f94006a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f4fa1200640 1 --2- 172.21.15.16:0/1067368118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f940e2c10 0x7f4f94006a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4f94055500 tx=0x7f4f880091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f4396000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f438800c040 0x7f43880340c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f4396000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4388053020 0x7f43880f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f4396000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43880ddd90 0x7f43880cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f4396000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43880f83a0 con 0x7f4388053020 2024-08-17T02:45:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f4396000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43880f85b0 con 0x7f438800c040 2024-08-17T02:45:26.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f4396000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43880f8890 con 0x7f43880ddd90 2024-08-17T02:45:26.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f4fa1200640 1 -- 172.21.15.16:0/1067368118 shutdown_connections 2024-08-17T02:45:26.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f4fa1200640 1 --2- 172.21.15.16:0/1067368118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f940e2c10 0x7f4f94006a80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f4fa1200640 1 --2- 172.21.15.16:0/1067368118 >> v2:172.21.15.142:3300/0 conn(0x7f4f940dc9b0 0x7f4f940e26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f4fa1200640 1 -- 172.21.15.16:0/1067368118 >> 172.21.15.16:0/1067368118 conn(0x7f4f94052710 msgr2=0x7f4f94052b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.558+0000 7f754f400640 1 -- start start 2024-08-17T02:45:26.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.559+0000 7f4394c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f438800c040 0x7f43880340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.559+0000 7f4394c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f438800c040 0x7f43880340c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36706/0 (socket says 172.21.15.16:36706) 2024-08-17T02:45:26.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.559+0000 7f4394c00640 1 -- 172.21.15.16:0/2114570094 learned_addr learned my addr 172.21.15.16:0/2114570094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.559+0000 7f4395600640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43880ddd90 0x7f43880cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.559+0000 7f438fe00640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4388053020 0x7f43880f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.559+0000 7f4394c00640 1 -- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43880ddd90 msgr2=0x7f43880cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.559+0000 7f4394c00640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43880ddd90 0x7f43880cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.559+0000 7f4394c00640 1 -- 172.21.15.16:0/2114570094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4388053020 msgr2=0x7f43880f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.559+0000 7f4394c00640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4388053020 0x7f43880f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.559+0000 7f4394c00640 1 -- 172.21.15.16:0/2114570094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4384009450 con 0x7f438800c040 2024-08-17T02:45:26.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.560+0000 7f4fa1200640 1 -- 172.21.15.16:0/1067368118 shutdown_connections 2024-08-17T02:45:26.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.560+0000 7fcd75200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd70158ef0 0x7fcd700afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.560+0000 7fcd75200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd700b0010 0x7fcd70069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.560+0000 7fcd75200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcd7006a410 0x7fcd700a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.560+0000 7fcd75200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd700b10c0 con 0x7fcd70158ef0 2024-08-17T02:45:26.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.560+0000 7fcd75200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd701592c0 con 0x7fcd7006a410 2024-08-17T02:45:26.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.560+0000 7fcd75200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7006a830 con 0x7fcd700b0010 2024-08-17T02:45:26.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.560+0000 7f4fa1200640 1 -- 172.21.15.16:0/1067368118 wait complete. 2024-08-17T02:45:26.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.560+0000 7f4394c00640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f438800c040 0x7f43880340c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4384009210 tx=0x7f4384032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.560+0000 7fcd6d400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd700b0010 0x7fcd70069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.561+0000 7fcd6d400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd700b0010 0x7fcd70069ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:53042/0 (socket says 172.21.15.16:53042) 2024-08-17T02:45:26.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.561+0000 7f754f400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f75480ada10 0x7f75480b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.560+0000 7f8a81e00640 1 --2- 172.21.15.16:0/293719413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a84158e20 0x7f8a840a98e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009910 tx=0x7f8a740091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62dacc8309e7d483 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.561+0000 7fcd6d400640 1 -- 172.21.15.16:0/159016152 learned_addr learned my addr 172.21.15.16:0/159016152 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.561+0000 7fcd6de00640 1 --2- 172.21.15.16:0/159016152 >> v2:172.21.15.16:3300/0 conn(0x7fcd70158ef0 0x7fcd700afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.561+0000 7f4383e00640 1 -- 172.21.15.16:0/2114570094 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4384032e00 con 0x7f438800c040 2024-08-17T02:45:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.561+0000 7f4383e00640 1 -- 172.21.15.16:0/2114570094 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4384000c40 con 0x7f438800c040 2024-08-17T02:45:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.561+0000 7f4383e00640 1 -- 172.21.15.16:0/2114570094 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4384031cb0 con 0x7f438800c040 2024-08-17T02:45:26.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.561+0000 7f754f400640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75480d0ab0 con 0x7f75480b1120 2024-08-17T02:45:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.561+0000 7f754f400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75480b00c0 con 0x7f75480b38b0 2024-08-17T02:45:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.561+0000 7f754f400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75480b0360 con 0x7f75480ada10 2024-08-17T02:45:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.562+0000 7fcd6de00640 1 -- 172.21.15.16:0/159016152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd700b0010 msgr2=0x7fcd70069ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.562+0000 7fcd6de00640 1 --2- 172.21.15.16:0/159016152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd700b0010 0x7fcd70069ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.562+0000 7fcd6de00640 1 -- 172.21.15.16:0/159016152 >> v2:172.21.15.142:3300/0 conn(0x7fcd7006a410 msgr2=0x7fcd700a98e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-17T02:45:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.562+0000 7fcd6de00640 1 --2- 172.21.15.16:0/159016152 >> v2:172.21.15.142:3300/0 conn(0x7fcd7006a410 0x7fcd700a98e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.562+0000 7fcd6de00640 1 -- 172.21.15.16:0/159016152 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd700b04b0 con 0x7fcd70158ef0 2024-08-17T02:45:26.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.562+0000 7f754ca00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f75480ada10 0x7f75480b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.562+0000 7f754ca00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f75480ada10 0x7f75480b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58022/0 (socket says 172.21.15.16:58022) 2024-08-17T02:45:26.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.562+0000 7f754ca00640 1 -- 172.21.15.16:0/2491259861 learned_addr learned my addr 172.21.15.16:0/2491259861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.562+0000 7f8a80a00640 1 -- 172.21.15.16:0/293719413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7402f510 con 0x7f8a84158e20 2024-08-17T02:45:26.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.562+0000 7f8a80a00640 1 -- 172.21.15.16:0/293719413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7402d050 con 0x7f8a84158e20 2024-08-17T02:45:26.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f4396000640 1 -- 172.21.15.16:0/2114570094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43880f8b70 con 0x7f438800c040 2024-08-17T02:45:26.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f8a80a00640 1 -- 172.21.15.16:0/293719413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7402f510 con 0x7f8a84158e20 2024-08-17T02:45:26.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4063608821 0 0) 0x7f75480b00c0 con 0x7f75480b38b0 2024-08-17T02:45:26.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f752c0034e0 con 0x7f75480b38b0 2024-08-17T02:45:26.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f4396000640 1 -- 172.21.15.16:0/2114570094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43880f9130 con 0x7f438800c040 2024-08-17T02:45:26.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3367269443 0 0) 0x7f752c0034e0 con 0x7f75480b38b0 2024-08-17T02:45:26.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 >> v2:172.21.15.16:3300/0 conn(0x7f75480ada10 msgr2=0x7f75480b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f7547400640 1 --2- 172.21.15.16:0/2491259861 >> v2:172.21.15.16:3300/0 conn(0x7f75480ada10 0x7f75480b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 >> v1:172.21.15.16:6789/0 conn(0x7f75480b1120 legacy=0x7f75480b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 --> v1:172.21.15.142:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75480b0640 con 0x7f75480b38b0 2024-08-17T02:45:26.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.564+0000 7f4fa1200640 1 Processor -- start 2024-08-17T02:45:26.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 <== mon.1 v1:172.21.15.142:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7f75400017f0 con 0x7f75480b38b0 2024-08-17T02:45:26.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 >> v1:172.21.15.142:6789/0 conn(0x7f75480b38b0 legacy=0x7f75480aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f9348c00640 1 Processor -- start 2024-08-17T02:45:26.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.563+0000 7f9348c00640 1 -- start start 2024-08-17T02:45:26.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.564+0000 7f7547400640 1 --2- 172.21.15.16:0/2491259861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f752c003af0 0x7f752c003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.564+0000 7f7547400640 1 --2- 172.21.15.16:0/2491259861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c004780 0x7f752c004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.564+0000 7f7547400640 1 --2- 172.21.15.16:0/2491259861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f752c005230 0x7f752c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.564+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75480b0360 con 0x7f752c004780 2024-08-17T02:45:26.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.564+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c00c150 con 0x7f752c003af0 2024-08-17T02:45:26.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.564+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c00c310 con 0x7f752c005230 2024-08-17T02:45:26.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.564+0000 7f754ca00640 1 --2- 172.21.15.16:0/2491259861 >> v2:172.21.15.16:3300/0 conn(0x7f75480ada10 0x7f75480b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:26.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.564+0000 7f4fa1200640 1 -- start start 2024-08-17T02:45:26.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.564+0000 7f4396000640 1 -- 172.21.15.16:0/2114570094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4388005320 con 0x7f438800c040 2024-08-17T02:45:26.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.565+0000 7f8a83200640 1 -- 172.21.15.16:0/293719413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a84158e20 msgr2=0x7f8a840a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.565+0000 7f8a83200640 1 --2- 172.21.15.16:0/293719413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a84158e20 0x7f8a840a98e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009910 tx=0x7f8a740091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.565+0000 7f9348c00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9344165d20 0x7f93440aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.565+0000 7f9348c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f93440b1710 0x7f934406a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.565+0000 7f754ca00640 1 --2- 172.21.15.16:0/2491259861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f752c003af0 0x7f752c003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.565+0000 7f7547e00640 1 --2- 172.21.15.16:0/2491259861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c004780 0x7f752c004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.565+0000 7f754d400640 1 --2- 172.21.15.16:0/2491259861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f752c005230 0x7f752c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f754ca00640 1 -- 172.21.15.16:0/2491259861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f752c005230 msgr2=0x7f752c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.565+0000 7f9348c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93440b1b90 con 0x7f9344069ed0 2024-08-17T02:45:26.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.565+0000 7f9348c00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934406a870 con 0x7f9344165d20 2024-08-17T02:45:26.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f9348c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9344166b00 con 0x7f93440b1710 2024-08-17T02:45:26.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f754ca00640 1 --2- 172.21.15.16:0/2491259861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f752c005230 0x7f752c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f9341400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f9344069ed0 0x7f9344166f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:34974/0 (socket says 172.21.15.16:34974) 2024-08-17T02:45:26.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f9341400640 1 -- 172.21.15.16:0/1165543170 learned_addr learned my addr 172.21.15.16:0/1165543170 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:26.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f4fa1200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f940dc5e0 0x7f4f94038180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f9342800640 1 --2- 172.21.15.16:0/1165543170 >> v2:172.21.15.142:3300/0 conn(0x7f93440b1710 0x7f934406a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f4fa1200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f940dc9b0 0x7f4f940f0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f4383e00640 1 -- 172.21.15.16:0/2114570094 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f4384039040 con 0x7f438800c040 2024-08-17T02:45:26.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f754ca00640 1 -- 172.21.15.16:0/2491259861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c004780 msgr2=0x7f752c004bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f754ca00640 1 --2- 172.21.15.16:0/2491259861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c004780 0x7f752c004bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f754ca00640 1 -- 172.21.15.16:0/2491259861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c0034e0 con 0x7f752c003af0 2024-08-17T02:45:26.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f754d400640 1 --2- 172.21.15.16:0/2491259861 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f752c005230 0x7f752c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:26.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f754d400640 1 -- 172.21.15.16:0/2491259861 reap_dead start 2024-08-17T02:45:26.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f4fa1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f94035370 0x7f4f940fa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f4fa1200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f940f9e80 con 0x7f4f94035370 2024-08-17T02:45:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f4f9b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f940dc5e0 0x7f4f94038180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.567+0000 7f4f9b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f940dc5e0 0x7f4f94038180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36724/0 (socket says 172.21.15.16:36724) 2024-08-17T02:45:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.567+0000 7f4f9b400640 1 -- 172.21.15.16:0/2032573780 learned_addr learned my addr 172.21.15.16:0/2032573780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.567+0000 7f4f9aa00640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f940dc9b0 0x7f4f940f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.567+0000 7f4fa1200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f940f9b80 con 0x7f4f940dc5e0 2024-08-17T02:45:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.567+0000 7f4fa1200640 1 -- 172.21.15.16:0/2032573780 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f94035b50 con 0x7f4f940dc9b0 2024-08-17T02:45:26.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.567+0000 7f4f9be00640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f94035370 0x7f4f940fa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f9341e00640 1 --2- 172.21.15.16:0/1165543170 >> v2:172.21.15.16:3300/0 conn(0x7f9344165d20 0x7f93440aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.567+0000 7f9341e00640 1 -- 172.21.15.16:0/1165543170 >> v2:172.21.15.142:3300/0 conn(0x7f93440b1710 msgr2=0x7f934406a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.567+0000 7f9341e00640 1 --2- 172.21.15.16:0/1165543170 >> v2:172.21.15.142:3300/0 conn(0x7f93440b1710 0x7f934406a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.567+0000 7f9341e00640 1 -- 172.21.15.16:0/1165543170 >> v1:172.21.15.142:6789/0 conn(0x7f9344069ed0 legacy=0x7f9344166f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.567+0000 7f9341e00640 1 -- 172.21.15.16:0/1165543170 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934406eac0 con 0x7f9344165d20 2024-08-17T02:45:26.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.567+0000 7fcd6de00640 1 --2- 172.21.15.16:0/159016152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70158ef0 0x7fcd700afad0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcd700b14f0 tx=0x7fcd68009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580daf62f4b5bc89 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.567+0000 7f4383e00640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f43700652a0 0x7f4370067750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.568+0000 7f4f9b400640 1 -- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f940dc9b0 msgr2=0x7f4f940f0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.568+0000 7f4383e00640 1 -- 172.21.15.16:0/2114570094 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f43840a20d0 con 0x7f438800c040 2024-08-17T02:45:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.568+0000 7f4f9b400640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f940dc9b0 0x7f4f940f0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.568+0000 7f754ca00640 1 --2- 172.21.15.16:0/2491259861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f752c003af0 0x7f752c003f00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7538009810 tx=0x7f75380091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76bae7ca76a56211 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.568+0000 7f4f9b400640 1 -- 172.21.15.16:0/2032573780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f94035370 msgr2=0x7f4f940fa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.568+0000 7f438fe00640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f43700652a0 0x7f4370067750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.568+0000 7f4f9b400640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f94035370 0x7f4f940fa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.568+0000 7f4f9b400640 1 -- 172.21.15.16:0/2032573780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f88009450 con 0x7f4f940dc5e0 2024-08-17T02:45:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.568+0000 7f4f9be00640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f94035370 0x7f4f940fa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:26.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.569+0000 7f9341e00640 1 --2- 172.21.15.16:0/1165543170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9344165d20 0x7f93440aea20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f93440b0650 tx=0x7f93340090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6ea2ef335c09a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.568+0000 7f4f9b400640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f940dc5e0 0x7f4f94038180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c04c0c0 tx=0x7f4f9c06f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.569+0000 7f438fe00640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f43700652a0 0x7f4370067750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4390065d00 tx=0x7f4390069040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.569+0000 7fcd6ca00640 1 -- 172.21.15.16:0/159016152 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68007550 con 0x7fcd70158ef0 2024-08-17T02:45:26.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f8a83200640 1 -- 172.21.15.16:0/293719413 shutdown_connections 2024-08-17T02:45:26.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f8a83200640 1 --2- 172.21.15.16:0/293719413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a84158e20 0x7f8a840a98e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.566+0000 7f8a83200640 1 -- 172.21.15.16:0/293719413 >> 172.21.15.16:0/293719413 conn(0x7f8a840cb790 msgr2=0x7f8a840cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.569+0000 7f9340a00640 1 -- 172.21.15.16:0/1165543170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933402ae40 con 0x7f9344165d20 2024-08-17T02:45:26.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.569+0000 7f9340a00640 1 -- 172.21.15.16:0/1165543170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9334032440 con 0x7f9344165d20 2024-08-17T02:45:26.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.569+0000 7f9340a00640 1 -- 172.21.15.16:0/1165543170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933402ae40 con 0x7f9344165d20 2024-08-17T02:45:26.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.569+0000 7fcd6ca00640 1 -- 172.21.15.16:0/159016152 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6802d050 con 0x7fcd70158ef0 2024-08-17T02:45:26.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7fcd6ca00640 1 -- 172.21.15.16:0/159016152 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68007920 con 0x7fcd70158ef0 2024-08-17T02:45:26.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7f8a83200640 1 -- 172.21.15.16:0/293719413 shutdown_connections 2024-08-17T02:45:26.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7f8a83200640 1 -- 172.21.15.16:0/293719413 wait complete. 2024-08-17T02:45:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7f6f95e00640 1 Processor -- start 2024-08-17T02:45:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7f9348c00640 1 -- 172.21.15.16:0/1165543170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9344165d20 msgr2=0x7f93440aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7f9348c00640 1 --2- 172.21.15.16:0/1165543170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9344165d20 0x7f93440aea20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f93440b0650 tx=0x7f93340090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7f6f95e00640 1 -- start start 2024-08-17T02:45:26.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7fcd75200640 1 -- 172.21.15.16:0/159016152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70158ef0 msgr2=0x7fcd700afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7f9348c00640 1 -- 172.21.15.16:0/1165543170 shutdown_connections 2024-08-17T02:45:26.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7f9348c00640 1 --2- 172.21.15.16:0/1165543170 >> v2:172.21.15.142:3300/0 conn(0x7f93440b1710 0x7f934406a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7f9348c00640 1 --2- 172.21.15.16:0/1165543170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9344165d20 0x7f93440aea20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7f9348c00640 1 -- 172.21.15.16:0/1165543170 >> 172.21.15.16:0/1165543170 conn(0x7f93440cc480 msgr2=0x7f93440687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7fcd75200640 1 --2- 172.21.15.16:0/159016152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70158ef0 0x7fcd700afad0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcd700b14f0 tx=0x7fcd68009780 comp rx=0 tx=0).stop 2024-08-17T02:45:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7fcd75200640 1 -- 172.21.15.16:0/159016152 shutdown_connections 2024-08-17T02:45:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7fcd75200640 1 --2- 172.21.15.16:0/159016152 >> v2:172.21.15.142:3300/0 conn(0x7fcd7006a410 0x7fcd700a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7fcd75200640 1 --2- 172.21.15.16:0/159016152 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd700b0010 0x7fcd70069ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.591 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7fcd75200640 1 --2- 172.21.15.16:0/159016152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd70158ef0 0x7fcd700afad0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.571+0000 7f9348c00640 1 -- 172.21.15.16:0/1165543170 shutdown_connections 2024-08-17T02:45:26.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.571+0000 7f9348c00640 1 -- 172.21.15.16:0/1165543170 wait complete. 2024-08-17T02:45:26.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.570+0000 7fcd75200640 1 -- 172.21.15.16:0/159016152 >> 172.21.15.16:0/159016152 conn(0x7fcd700cc480 msgr2=0x7fcd700687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.571+0000 7fcd75200640 1 -- 172.21.15.16:0/159016152 shutdown_connections 2024-08-17T02:45:26.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.571+0000 7fcd75200640 1 -- 172.21.15.16:0/159016152 wait complete. 2024-08-17T02:45:26.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f8a83200640 1 Processor -- start 2024-08-17T02:45:26.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f6f95e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f900cd700 0x7f6f900ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f6f95e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6f900cc9d0 0x7f6f900acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f6f95e00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f900cf9d0 con 0x7f6f90158c80 2024-08-17T02:45:26.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538033750 con 0x7f752c003af0 2024-08-17T02:45:26.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f8a83200640 1 -- start start 2024-08-17T02:45:26.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75380338b0 con 0x7f752c003af0 2024-08-17T02:45:26.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f7547400640 1 -- 172.21.15.16:0/2491259861 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538033c50 con 0x7f752c003af0 2024-08-17T02:45:26.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f4f8ea00640 1 -- 172.21.15.16:0/2032573780 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c06f470 con 0x7f4f940dc5e0 2024-08-17T02:45:26.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f4f8ea00640 1 -- 172.21.15.16:0/2032573780 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c08f3b0 con 0x7f4f940dc5e0 2024-08-17T02:45:26.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f4f8ea00640 1 -- 172.21.15.16:0/2032573780 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c08f770 con 0x7f4f940dc5e0 2024-08-17T02:45:26.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f4fa1200640 1 -- 172.21.15.16:0/2032573780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f94035e30 con 0x7f4f940dc5e0 2024-08-17T02:45:26.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.573+0000 7f4fa1200640 1 -- 172.21.15.16:0/2032573780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f940fa850 con 0x7f4f940dc5e0 2024-08-17T02:45:26.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f6f95e00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f900ceb60 con 0x7f6f900cd700 2024-08-17T02:45:26.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f6f95e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f900cedc0 con 0x7f6f900cc9d0 2024-08-17T02:45:26.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f6f8fe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f900cd700 0x7f6f900ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f6f8fe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f900cd700 0x7f6f900ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58072/0 (socket says 172.21.15.16:58072) 2024-08-17T02:45:26.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.572+0000 7f6f8fe00640 1 -- 172.21.15.16:0/2496670501 learned_addr learned my addr 172.21.15.16:0/2496670501 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.573+0000 7f754f400640 1 -- 172.21.15.16:0/2491259861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f752c003af0 msgr2=0x7f752c003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.573+0000 7f754f400640 1 --2- 172.21.15.16:0/2491259861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f752c003af0 0x7f752c003f00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7538009810 tx=0x7f75380091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.573+0000 7f4383e00640 1 -- 172.21.15.16:0/2114570094 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4384070220 con 0x7f438800c040 2024-08-17T02:45:26.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.574+0000 7f8a83200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a840afed0 0x7f8a8416a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.575+0000 7f8a83200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a84158e20 0x7f8a84167240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.574+0000 7f754f400640 1 -- 172.21.15.16:0/2491259861 shutdown_connections 2024-08-17T02:45:26.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.574+0000 7f754f400640 1 --2- 172.21.15.16:0/2491259861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f752c003af0 0x7f752c003f00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.574+0000 7f754f400640 1 -- 172.21.15.16:0/2491259861 >> 172.21.15.16:0/2491259861 conn(0x7f75480cfdf0 msgr2=0x7f75480d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.573+0000 7f9348c00640 1 Processor -- start 2024-08-17T02:45:26.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.574+0000 7f6f95400640 1 --2- 172.21.15.16:0/2496670501 >> v2:172.21.15.142:3300/0 conn(0x7f6f900cc9d0 0x7f6f900acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.575+0000 7f6f8fe00640 1 -- 172.21.15.16:0/2496670501 >> v2:172.21.15.142:3300/0 conn(0x7f6f900cc9d0 msgr2=0x7f6f900acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:26.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.575+0000 7f6f8fe00640 1 --2- 172.21.15.16:0/2496670501 >> v2:172.21.15.142:3300/0 conn(0x7f6f900cc9d0 0x7f6f900acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.575+0000 7f6f8fe00640 1 -- 172.21.15.16:0/2496670501 >> v1:172.21.15.16:6789/0 conn(0x7f6f90158c80 legacy=0x7f6f900cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.574+0000 7f4fa1200640 1 -- 172.21.15.16:0/2032573780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f94005770 con 0x7f4f940dc5e0 2024-08-17T02:45:26.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.575+0000 7f4f8ea00640 1 -- 172.21.15.16:0/2032573780 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f4f9c08fb60 con 0x7f4f940dc5e0 2024-08-17T02:45:26.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.575+0000 7f8a83200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a841591f0 0x7f8a840c18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.575+0000 7fcd75200640 1 Processor -- start 2024-08-17T02:45:26.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.575+0000 7f8a83200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8416b9c0 con 0x7f8a84158e20 2024-08-17T02:45:26.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.575+0000 7f8a83200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8416adb0 con 0x7f8a841591f0 2024-08-17T02:45:26.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.576+0000 7f8a83200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8416b090 con 0x7f8a840afed0 2024-08-17T02:45:26.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.576+0000 7f8a81e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a840afed0 0x7f8a8416a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.576+0000 7f8a81e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a840afed0 0x7f8a8416a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:53068/0 (socket says 172.21.15.16:53068) 2024-08-17T02:45:26.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.576+0000 7f8a81e00640 1 -- 172.21.15.16:0/2397292954 learned_addr learned my addr 172.21.15.16:0/2397292954 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.576+0000 7f8a82800640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a841591f0 0x7f8a840c18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.576+0000 7f4f8ea00640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4f7c065220 0x7f4f7c0676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.576+0000 7f8a81400640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a84158e20 0x7f8a84167240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.576+0000 7f4f8ea00640 1 -- 172.21.15.16:0/2032573780 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4f9c0f8e60 con 0x7f4f940dc5e0 2024-08-17T02:45:26.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.575+0000 7f6f8fe00640 1 -- 172.21.15.16:0/2496670501 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f900cf0a0 con 0x7f6f900cd700 2024-08-17T02:45:26.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.575+0000 7f6f8f400640 1 -- 172.21.15.16:0/2496670501 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330624020 0 0) 0x7f6f900cf9d0 con 0x7f6f90158c80 2024-08-17T02:45:26.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.576+0000 7ff47e600640 1 Processor -- start 2024-08-17T02:45:26.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.576+0000 7fcd75200640 1 -- start start 2024-08-17T02:45:26.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.576+0000 7ff47e600640 1 -- start start 2024-08-17T02:45:26.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f8a82800640 1 -- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a840afed0 msgr2=0x7f8a8416a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f8a82800640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a840afed0 0x7f8a8416a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f8a82800640 1 -- 172.21.15.16:0/2397292954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a84158e20 msgr2=0x7f8a84167240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f8a82800640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a84158e20 0x7f8a84167240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f8a82800640 1 -- 172.21.15.16:0/2397292954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a74009450 con 0x7f8a841591f0 2024-08-17T02:45:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f6f8fe00640 1 --2- 172.21.15.16:0/2496670501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f900cd700 0x7f6f900ce620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6f900ca0f0 tx=0x7f6f7c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29e2dadf31bb0a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f4f9aa00640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4f7c065220 0x7f4f7c0676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.576+0000 7f9348c00640 1 -- start start 2024-08-17T02:45:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f8a82800640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a841591f0 0x7f8a840c18d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00c7c0 tx=0x7f8a7c0015e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f6f8f400640 1 -- 172.21.15.16:0/2496670501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c02adf0 con 0x7f6f900cd700 2024-08-17T02:45:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f754f400640 1 -- 172.21.15.16:0/2491259861 shutdown_connections 2024-08-17T02:45:26.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f6f8f400640 1 -- 172.21.15.16:0/2496670501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c032440 con 0x7f6f900cd700 2024-08-17T02:45:26.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f6f8f400640 1 -- 172.21.15.16:0/2496670501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0327e0 con 0x7f6f900cd700 2024-08-17T02:45:26.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.577+0000 7f754f400640 1 -- 172.21.15.16:0/2491259861 wait complete. 2024-08-17T02:45:26.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.578+0000 7f4f9aa00640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4f7c065220 0x7f4f7c0676d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f9403b7e0 tx=0x7f4f90009060 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.578+0000 7ff47e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4780d08d0 0x7ff4780c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.578+0000 7ff47e600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff4780cb340 0x7ff4781590b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.578+0000 7ff47e600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4780cb7c0 con 0x7ff478158ce0 2024-08-17T02:45:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.578+0000 7f9348c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9344069ed0 0x7f934415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.578+0000 7f9348c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93440b1710 0x7f9344163610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.578+0000 7ff47e600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4780cf0c0 con 0x7ff4780cb340 2024-08-17T02:45:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.578+0000 7f9348c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9344165d20 0x7f9344163b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7f6f95e00640 1 -- 172.21.15.16:0/2496670501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f900cd700 msgr2=0x7f6f900ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7f6f95e00640 1 --2- 172.21.15.16:0/2496670501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f900cd700 0x7f6f900ce620 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6f900ca0f0 tx=0x7f6f7c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.578+0000 7ff47e600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff478089000 con 0x7ff4780d08d0 2024-08-17T02:45:26.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.578+0000 7ff477400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7ff478158ce0 0x7ff4780cac30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:34984/0 (socket says 172.21.15.16:34984) 2024-08-17T02:45:26.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.578+0000 7ff477400640 1 -- 172.21.15.16:0/684255957 learned_addr learned my addr 172.21.15.16:0/684255957 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:26.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.578+0000 7ff46fe00640 1 --2- 172.21.15.16:0/684255957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4780d08d0 0x7ff4780c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7ff46fe00640 1 -- 172.21.15.16:0/684255957 >> v2:172.21.15.142:3300/0 conn(0x7ff4780cb340 msgr2=0x7ff4781590b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-17T02:45:26.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7ff46fe00640 1 --2- 172.21.15.16:0/684255957 >> v2:172.21.15.142:3300/0 conn(0x7ff4780cb340 0x7ff4781590b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7ff46fe00640 1 -- 172.21.15.16:0/684255957 >> v1:172.21.15.142:6789/0 conn(0x7ff478158ce0 legacy=0x7ff4780cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7ff46fe00640 1 -- 172.21.15.16:0/684255957 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4780ceb20 con 0x7ff4780d08d0 2024-08-17T02:45:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7f6f95e00640 1 -- 172.21.15.16:0/2496670501 shutdown_connections 2024-08-17T02:45:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7f6f95e00640 1 --2- 172.21.15.16:0/2496670501 >> v2:172.21.15.142:3300/0 conn(0x7f6f900cc9d0 0x7f6f900acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7fcd75200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd7006a410 0x7fcd7014b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7f6f95e00640 1 --2- 172.21.15.16:0/2496670501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f900cd700 0x7f6f900ce620 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7f6f95e00640 1 -- 172.21.15.16:0/2496670501 >> 172.21.15.16:0/2496670501 conn(0x7f6f90098e30 msgr2=0x7f6f9006eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7fcd75200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd700b0010 0x7fcd70154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7f9348c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934415ede0 con 0x7f93440b1710 2024-08-17T02:45:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7f9348c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934415f080 con 0x7f9344165d20 2024-08-17T02:45:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7f9348c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934415f360 con 0x7f9344069ed0 2024-08-17T02:45:26.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7f9341400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93440b1710 0x7f9344163610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7f9341400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93440b1710 0x7f9344163610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58096/0 (socket says 172.21.15.16:58096) 2024-08-17T02:45:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.579+0000 7f9341400640 1 -- 172.21.15.16:0/4271043779 learned_addr learned my addr 172.21.15.16:0/4271043779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7f754f400640 1 Processor -- start 2024-08-17T02:45:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7f9341e00640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9344069ed0 0x7f934415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7fcd6de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd7006a410 0x7fcd7014b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7f754f400640 1 -- start start 2024-08-17T02:45:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7f6f95e00640 1 -- 172.21.15.16:0/2496670501 shutdown_connections 2024-08-17T02:45:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7fcd6de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd7006a410 0x7fcd7014b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:53074/0 (socket says 172.21.15.16:53074) 2024-08-17T02:45:26.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7fcd75200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd701554f0 0x7fcd700ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7f6f95e00640 1 -- 172.21.15.16:0/2496670501 wait complete. 2024-08-17T02:45:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7fcd6de00640 1 -- 172.21.15.16:0/2073881829 learned_addr learned my addr 172.21.15.16:0/2073881829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70155990 con 0x7fcd701554f0 2024-08-17T02:45:26.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7015b980 con 0x7fcd700b0010 2024-08-17T02:45:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7ff46fe00640 1 --2- 172.21.15.16:0/684255957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4780d08d0 0x7ff4780c68c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff460009850 tx=0x7ff4600090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c9f680d5855689a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7015adb0 con 0x7fcd7006a410 2024-08-17T02:45:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7fcd6e800640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd701554f0 0x7fcd700ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7f9342800640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9344165d20 0x7f9344163b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7f8a79600640 1 -- 172.21.15.16:0/2397292954 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c004b20 con 0x7f8a841591f0 2024-08-17T02:45:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7fcd6de00640 1 -- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd700b0010 msgr2=0x7fcd70154fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-17T02:45:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7fcd6de00640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd700b0010 0x7fcd70154fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7fcd6de00640 1 -- 172.21.15.16:0/2073881829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd701554f0 msgr2=0x7fcd700ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7fcd6de00640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd701554f0 0x7fcd700ac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7fcd6de00640 1 -- 172.21.15.16:0/2073881829 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68009a20 con 0x7fcd7006a410 2024-08-17T02:45:26.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7fcd6e800640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd701554f0 0x7fcd700ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7f9341e00640 1 -- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9344165d20 msgr2=0x7f9344163b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7f8a79600640 1 -- 172.21.15.16:0/2397292954 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c004ca0 con 0x7f8a841591f0 2024-08-17T02:45:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7f8a83200640 1 -- 172.21.15.16:0/2397292954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8416b370 con 0x7f8a841591f0 2024-08-17T02:45:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7f8a79600640 1 -- 172.21.15.16:0/2397292954 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c00a890 con 0x7f8a841591f0 2024-08-17T02:45:26.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7f8a83200640 1 -- 172.21.15.16:0/2397292954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a84167780 con 0x7f8a841591f0 2024-08-17T02:45:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7f9341e00640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9344165d20 0x7f9344163b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.582+0000 7fcd6de00640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd7006a410 0x7fcd7014b760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcd680099f0 tx=0x7fcd68037000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7f9341e00640 1 -- 172.21.15.16:0/4271043779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93440b1710 msgr2=0x7f9344163610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7f9341e00640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93440b1710 0x7f9344163610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7f9341e00640 1 -- 172.21.15.16:0/4271043779 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334009390 con 0x7f9344069ed0 2024-08-17T02:45:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.580+0000 7f9341400640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93440b1710 0x7f9344163610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7f9341e00640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9344069ed0 0x7f934415af20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9334031000 tx=0x7f9334032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7f9339600640 1 -- 172.21.15.16:0/4271043779 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340317a0 con 0x7f9344069ed0 2024-08-17T02:45:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7f9339600640 1 -- 172.21.15.16:0/4271043779 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9334041070 con 0x7f9344069ed0 2024-08-17T02:45:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.581+0000 7f9339600640 1 -- 172.21.15.16:0/4271043779 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334031b90 con 0x7f9344069ed0 2024-08-17T02:45:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.582+0000 7f4f8ea00640 1 -- 172.21.15.16:0/2032573780 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4f9c0c6fb0 con 0x7f4f940dc5e0 2024-08-17T02:45:26.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.582+0000 7ff476a00640 1 -- 172.21.15.16:0/684255957 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46002adb0 con 0x7ff4780d08d0 2024-08-17T02:45:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.582+0000 7ff476a00640 1 -- 172.21.15.16:0/684255957 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff460032440 con 0x7ff4780d08d0 2024-08-17T02:45:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.582+0000 7ff476a00640 1 -- 172.21.15.16:0/684255957 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4600327e0 con 0x7ff4780d08d0 2024-08-17T02:45:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.582+0000 7fcd65600640 1 -- 172.21.15.16:0/2073881829 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68001040 con 0x7fcd7006a410 2024-08-17T02:45:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.582+0000 7fcd65600640 1 -- 172.21.15.16:0/2073881829 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd68040070 con 0x7fcd7006a410 2024-08-17T02:45:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.582+0000 7fcd65600640 1 -- 172.21.15.16:0/2073881829 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68031b00 con 0x7fcd7006a410 2024-08-17T02:45:26.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.582+0000 7f6f95e00640 1 Processor -- start 2024-08-17T02:45:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.583+0000 7f8a83200640 1 -- 172.21.15.16:0/2397292954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a84068640 con 0x7f8a841591f0 2024-08-17T02:45:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.583+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7015b050 con 0x7fcd7006a410 2024-08-17T02:45:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.583+0000 7ff47e600640 1 -- 172.21.15.16:0/684255957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4780d08d0 msgr2=0x7ff4780c68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.584+0000 7ff47e600640 1 --2- 172.21.15.16:0/684255957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4780d08d0 0x7ff4780c68c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff460009850 tx=0x7ff4600090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.584+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7015b610 con 0x7fcd7006a410 2024-08-17T02:45:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.584+0000 7ff47e600640 1 -- 172.21.15.16:0/684255957 shutdown_connections 2024-08-17T02:45:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.584+0000 7ff47e600640 1 --2- 172.21.15.16:0/684255957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4780d08d0 0x7ff4780c68c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.584+0000 7ff47e600640 1 --2- 172.21.15.16:0/684255957 >> v2:172.21.15.142:3300/0 conn(0x7ff4780cb340 0x7ff4781590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.584+0000 7ff47e600640 1 -- 172.21.15.16:0/684255957 >> 172.21.15.16:0/684255957 conn(0x7ff4780cfdc0 msgr2=0x7ff4780d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.583+0000 7f9348c00640 1 -- 172.21.15.16:0/4271043779 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934415f640 con 0x7f9344069ed0 2024-08-17T02:45:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.583+0000 7f9348c00640 1 -- 172.21.15.16:0/4271043779 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9344158450 con 0x7f9344069ed0 2024-08-17T02:45:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.584+0000 7f9348c00640 1 -- 172.21.15.16:0/4271043779 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f934415f1c0 con 0x7f9344069ed0 2024-08-17T02:45:26.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f8a79600640 1 -- 172.21.15.16:0/2397292954 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f8a7c00ac40 con 0x7f8a841591f0 2024-08-17T02:45:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f9339600640 1 -- 172.21.15.16:0/4271043779 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f9334043030 con 0x7f9344069ed0 2024-08-17T02:45:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f9339600640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9320065150 0x7f9320067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f9339600640 1 -- 172.21.15.16:0/4271043779 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f933402ae40 con 0x7f9344069ed0 2024-08-17T02:45:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.583+0000 7f6f95e00640 1 -- start start 2024-08-17T02:45:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.584+0000 7f6f95e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f900cc9d0 0x7f6f900c3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.584+0000 7f6f95e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f900cd700 0x7f6f90168de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.584+0000 7f6f95e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f90158c80 0x7f6f90169320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f6f95e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f900c4a50 con 0x7f6f900cc9d0 2024-08-17T02:45:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f6f95e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f900c3e40 con 0x7f6f900cd700 2024-08-17T02:45:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f6f95e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f900c4120 con 0x7f6f90158c80 2024-08-17T02:45:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f6f8fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f900cd700 0x7f6f90168de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f6f8fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f900cd700 0x7f6f90168de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36760/0 (socket says 172.21.15.16:36760) 2024-08-17T02:45:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f6f8fe00640 1 -- 172.21.15.16:0/1646876883 learned_addr learned my addr 172.21.15.16:0/1646876883 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f6f8fe00640 1 -- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f90158c80 msgr2=0x7f6f90169320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-17T02:45:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f6f8fe00640 1 --2- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f90158c80 0x7f6f90169320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f6f8fe00640 1 -- 172.21.15.16:0/1646876883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f900cc9d0 msgr2=0x7f6f900c3610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-17T02:45:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f6f8fe00640 1 --2- 172.21.15.16:0/1646876883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f900cc9d0 0x7f6f900c3610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.585+0000 7f6f8fe00640 1 -- 172.21.15.16:0/1646876883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c009450 con 0x7f6f900cd700 2024-08-17T02:45:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f6f8fe00640 1 --2- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f900cd700 0x7f6f90168de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c035040 tx=0x7f6f7c032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f8a79600640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8a640652a0 0x7f8a64067750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f8a79600640 1 -- 172.21.15.16:0/2397292954 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8a7c081a30 con 0x7f8a841591f0 2024-08-17T02:45:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7ff47e600640 1 -- 172.21.15.16:0/684255957 shutdown_connections 2024-08-17T02:45:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f754f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c003af0 0x7f754809ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f754f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548164e40 0x7f7548099d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f754f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f754809a270 0x7f7548153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f754f400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754809c250 con 0x7f752c003af0 2024-08-17T02:45:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f754f400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754809c6e0 con 0x7f7548164e40 2024-08-17T02:45:26.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f754f400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754809b370 con 0x7f754809a270 2024-08-17T02:45:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f754ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c003af0 0x7f754809ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f7547e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548164e40 0x7f7548099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f7547e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548164e40 0x7f7548099d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36774/0 (socket says 172.21.15.16:36774) 2024-08-17T02:45:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.586+0000 7f7547e00640 1 -- 172.21.15.16:0/829799436 learned_addr learned my addr 172.21.15.16:0/829799436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.587+0000 7fcd65600640 1 -- 172.21.15.16:0/2073881829 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fcd68042030 con 0x7fcd7006a410 2024-08-17T02:45:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.587+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd70064db0 con 0x7fcd7006a410 2024-08-17T02:45:26.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.587+0000 7f9341400640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9320065150 0x7f9320067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.587+0000 7f8a81e00640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8a640652a0 0x7f8a64067750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.587+0000 7f7547e00640 1 -- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f754809a270 msgr2=0x7f7548153450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-17T02:45:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.587+0000 7f7547e00640 1 --2- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f754809a270 0x7f7548153450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.587+0000 7f7547e00640 1 -- 172.21.15.16:0/829799436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c003af0 msgr2=0x7f754809ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.587+0000 7f7547e00640 1 --2- 172.21.15.16:0/829799436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c003af0 0x7f754809ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.587+0000 7f7547e00640 1 -- 172.21.15.16:0/829799436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754809b640 con 0x7f7548164e40 2024-08-17T02:45:26.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.588+0000 7f9341400640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9320065150 0x7f9320067600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9344168490 tx=0x7f932c000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.588+0000 7f7547e00640 1 --2- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548164e40 0x7f7548099d30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f753400e470 tx=0x7f7534009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.588+0000 7f8a81e00640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8a640652a0 0x7f8a64067750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009420 tx=0x7f8a74035000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.587+0000 7ff47e600640 1 -- 172.21.15.16:0/684255957 wait complete. 2024-08-17T02:45:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.588+0000 7ff47e600640 1 Processor -- start 2024-08-17T02:45:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.589+0000 7ff47e600640 1 -- start start 2024-08-17T02:45:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.589+0000 7fcd65600640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcd50065220 0x7fcd500676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.589+0000 7fcd6d400640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcd50065220 0x7fcd500676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7ff47e600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4780cb340 0x7ff478141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7ff47e600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4780d08d0 0x7ff478156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7fcd6d400640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcd50065220 0x7fcd500676d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcd701529f0 tx=0x7fcd600091d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7f753fe00640 1 -- 172.21.15.16:0/829799436 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7534002030 con 0x7f7548164e40 2024-08-17T02:45:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7f6f83e00640 1 -- 172.21.15.16:0/1646876883 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c031a60 con 0x7f6f900cd700 2024-08-17T02:45:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7ff47e600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff478158ce0 0x7ff4781573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7ff46fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4780d08d0 0x7ff478156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7f753fe00640 1 -- 172.21.15.16:0/829799436 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7534013420 con 0x7f7548164e40 2024-08-17T02:45:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7f754f400640 1 -- 172.21.15.16:0/829799436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754809b920 con 0x7f7548164e40 2024-08-17T02:45:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7f8a79600640 1 -- 172.21.15.16:0/2397292954 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8a84068640 con 0x7f8a841591f0 2024-08-17T02:45:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7f6f83e00640 1 -- 172.21.15.16:0/1646876883 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c031bc0 con 0x7f6f900cd700 2024-08-17T02:45:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7ff46fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4780d08d0 0x7ff478156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58114/0 (socket says 172.21.15.16:58114) 2024-08-17T02:45:26.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4780b2e90 con 0x7ff4780d08d0 2024-08-17T02:45:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7ff46fe00640 1 -- 172.21.15.16:0/3357845269 learned_addr learned my addr 172.21.15.16:0/3357845269 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.591+0000 7f6f95e00640 1 -- 172.21.15.16:0/1646876883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f900c4400 con 0x7f6f900cd700 2024-08-17T02:45:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4780c8430 con 0x7ff4780cb340 2024-08-17T02:45:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.590+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4780c7ac0 con 0x7ff478158ce0 2024-08-17T02:45:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.591+0000 7ff477e00640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff478158ce0 0x7ff4781573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.591+0000 7f6f83e00640 1 -- 172.21.15.16:0/1646876883 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c03d420 con 0x7f6f900cd700 2024-08-17T02:45:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.591+0000 7f754f400640 1 -- 172.21.15.16:0/829799436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754809bee0 con 0x7f7548164e40 2024-08-17T02:45:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.591+0000 7f753fe00640 1 -- 172.21.15.16:0/829799436 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75340137c0 con 0x7f7548164e40 2024-08-17T02:45:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.591+0000 7f6f95e00640 1 -- 172.21.15.16:0/1646876883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f900c3b50 con 0x7f6f900cd700 2024-08-17T02:45:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.592+0000 7f9339600640 1 -- 172.21.15.16:0/4271043779 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f934415f1c0 con 0x7f9344069ed0 2024-08-17T02:45:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.592+0000 7ff46fe00640 1 -- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff478158ce0 msgr2=0x7ff4781573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.592+0000 7ff46fe00640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff478158ce0 0x7ff4781573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.592+0000 7f6f95e00640 1 -- 172.21.15.16:0/1646876883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f90071050 con 0x7f6f900cd700 2024-08-17T02:45:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.592+0000 7ff46fe00640 1 -- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4780cb340 msgr2=0x7ff478141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.592+0000 7ff46fe00640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4780cb340 0x7ff478141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.592+0000 7ff46fe00640 1 -- 172.21.15.16:0/3357845269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff460009390 con 0x7ff4780d08d0 2024-08-17T02:45:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.593+0000 7fcd65600640 1 -- 172.21.15.16:0/2073881829 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcd680759f0 con 0x7fcd7006a410 2024-08-17T02:45:26.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.593+0000 7ff46fe00640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4780d08d0 0x7ff478156e80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4600098b0 tx=0x7ff460032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.593+0000 7f754f400640 1 -- 172.21.15.16:0/829799436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7548064db0 con 0x7f7548164e40 2024-08-17T02:45:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.593+0000 7ff46ea00640 1 -- 172.21.15.16:0/3357845269 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460031a60 con 0x7ff4780d08d0 2024-08-17T02:45:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.593+0000 7ff46ea00640 1 -- 172.21.15.16:0/3357845269 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff460031bc0 con 0x7ff4780d08d0 2024-08-17T02:45:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.593+0000 7ff46ea00640 1 -- 172.21.15.16:0/3357845269 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46003c410 con 0x7ff4780d08d0 2024-08-17T02:45:26.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.595+0000 7f6f83e00640 1 -- 172.21.15.16:0/1646876883 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f6f7c039040 con 0x7f6f900cd700 2024-08-17T02:45:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.594+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4780c7da0 con 0x7ff4780d08d0 2024-08-17T02:45:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.595+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4780c8960 con 0x7ff4780d08d0 2024-08-17T02:45:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.595+0000 7f753fe00640 1 -- 172.21.15.16:0/829799436 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f7534013b30 con 0x7f7548164e40 2024-08-17T02:45:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.597+0000 7ff46ea00640 1 -- 172.21.15.16:0/3357845269 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7ff460039050 con 0x7ff4780d08d0 2024-08-17T02:45:26.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.597+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4780c8bb0 con 0x7ff4780d08d0 2024-08-17T02:45:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.596+0000 7f753fe00640 1 --2- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7528065150 0x7f7528067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.596+0000 7f753fe00640 1 -- 172.21.15.16:0/829799436 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7534003030 con 0x7f7548164e40 2024-08-17T02:45:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.597+0000 7f754ca00640 1 --2- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7528065150 0x7f7528067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.598+0000 7ff46ea00640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff454065220 0x7ff4540676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.599+0000 7f754ca00640 1 --2- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7528065150 0x7f7528067600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f753803d000 tx=0x7f753803b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.598+0000 7ff46ea00640 1 -- 172.21.15.16:0/3357845269 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff4600a22e0 con 0x7ff4780d08d0 2024-08-17T02:45:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.599+0000 7fcd65600640 1 -- 172.21.15.16:0/2073881829 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcd68071630 con 0x7fcd7006a410 2024-08-17T02:45:26.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.600+0000 7f753fe00640 1 -- 172.21.15.16:0/829799436 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f753404fd80 con 0x7f7548164e40 2024-08-17T02:45:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.600+0000 7f6f83e00640 1 --2- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6f70065150 0x7f6f70067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.600+0000 7f6f94a00640 1 --2- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6f70065150 0x7f6f70067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.600+0000 7f6f83e00640 1 -- 172.21.15.16:0/1646876883 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f7c0746b0 con 0x7f6f900cd700 2024-08-17T02:45:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.602+0000 7f6f94a00640 1 --2- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6f70065150 0x7f6f70067600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f84007fd0 tx=0x7f6f840078d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.603+0000 7ff477400640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff454065220 0x7ff4540676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.603+0000 7ff477400640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff454065220 0x7ff4540676d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff468007fd0 tx=0x7ff468007b00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.605+0000 7f6f83e00640 1 -- 172.21.15.16:0/1646876883 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6f7c0702f0 con 0x7f6f900cd700 2024-08-17T02:45:26.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:26.605+0000 7ff46ea00640 1 -- 172.21.15.16:0/3357845269 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff460070390 con 0x7ff4780d08d0 2024-08-17T02:45:27.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.002+0000 7f4fa1200640 1 -- 172.21.15.16:0/2032573780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4f94034760 con 0x7f4f940dc5e0 2024-08-17T02:45:27.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.003+0000 7f4f8ea00640 1 -- 172.21.15.16:0/2032573780 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4f9c095090 con 0x7f4f940dc5e0 2024-08-17T02:45:27.007 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:27.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.005+0000 7f4396000640 1 -- 172.21.15.16:0/2114570094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4388052e00 con 0x7f438800c040 2024-08-17T02:45:27.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.006+0000 7f4383e00640 1 -- 172.21.15.16:0/2114570094 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4384037020 con 0x7f438800c040 2024-08-17T02:45:27.008 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2024-08-17T02:45:27.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.007+0000 7f4fa1200640 1 -- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4f7c065220 msgr2=0x7f4f7c0676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.007+0000 7f4fa1200640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4f7c065220 0x7f4f7c0676d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f9403b7e0 tx=0x7f4f90009060 comp rx=0 tx=0).stop 2024-08-17T02:45:27.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.007+0000 7f4fa1200640 1 -- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f940dc5e0 msgr2=0x7f4f94038180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.007+0000 7f4fa1200640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f940dc5e0 0x7f4f94038180 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c04c0c0 tx=0x7f4f9c06f710 comp rx=0 tx=0).stop 2024-08-17T02:45:27.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.007+0000 7f4fa1200640 1 -- 172.21.15.16:0/2032573780 shutdown_connections 2024-08-17T02:45:27.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.007+0000 7f4fa1200640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4f7c065220 0x7f4f7c0676d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.007+0000 7f4fa1200640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f94035370 0x7f4f940fa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.007+0000 7f4fa1200640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f940dc9b0 0x7f4f940f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.007+0000 7f8a83200640 1 -- 172.21.15.16:0/2397292954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8a84068640 con 0x7f8a841591f0 2024-08-17T02:45:27.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.007+0000 7f4fa1200640 1 --2- 172.21.15.16:0/2032573780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f940dc5e0 0x7f4f94038180 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.007+0000 7f4fa1200640 1 -- 172.21.15.16:0/2032573780 >> 172.21.15.16:0/2032573780 conn(0x7f4f94052710 msgr2=0x7f4f940e4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:27.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.008+0000 7f4fa1200640 1 -- 172.21.15.16:0/2032573780 shutdown_connections 2024-08-17T02:45:27.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.008+0000 7f8a79600640 1 -- 172.21.15.16:0/2397292954 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8a84068640 con 0x7f8a841591f0 2024-08-17T02:45:27.011 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:27.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.008+0000 7f4fa1200640 1 -- 172.21.15.16:0/2032573780 wait complete. 2024-08-17T02:45:27.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.011+0000 7f4396000640 1 -- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f43700652a0 msgr2=0x7f4370067750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.011+0000 7f4396000640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f43700652a0 0x7f4370067750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4390065d00 tx=0x7f4390069040 comp rx=0 tx=0).stop 2024-08-17T02:45:27.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.011+0000 7f8a83200640 1 -- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8a640652a0 msgr2=0x7f8a64067750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.011+0000 7f8a83200640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8a640652a0 0x7f8a64067750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009420 tx=0x7f8a74035000 comp rx=0 tx=0).stop 2024-08-17T02:45:27.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.011+0000 7f8a83200640 1 -- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a841591f0 msgr2=0x7f8a840c18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.011+0000 7f8a83200640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a841591f0 0x7f8a840c18d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00c7c0 tx=0x7f8a7c0015e0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.011+0000 7f4396000640 1 -- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f438800c040 msgr2=0x7f43880340c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f4396000640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f438800c040 0x7f43880340c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4384009210 tx=0x7f4384032000 comp rx=0 tx=0).stop 2024-08-17T02:45:27.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f8a83200640 1 -- 172.21.15.16:0/2397292954 shutdown_connections 2024-08-17T02:45:27.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f8a83200640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8a640652a0 0x7f8a64067750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f8a83200640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8a841591f0 0x7f8a840c18d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f8a83200640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a84158e20 0x7f8a84167240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f8a83200640 1 --2- 172.21.15.16:0/2397292954 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8a840afed0 0x7f8a8416a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f8a83200640 1 -- 172.21.15.16:0/2397292954 >> 172.21.15.16:0/2397292954 conn(0x7f8a840cb790 msgr2=0x7f8a840ac5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:27.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f4396000640 1 -- 172.21.15.16:0/2114570094 shutdown_connections 2024-08-17T02:45:27.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f8a83200640 1 -- 172.21.15.16:0/2397292954 shutdown_connections 2024-08-17T02:45:27.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f4396000640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f43700652a0 0x7f4370067750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f8a83200640 1 -- 172.21.15.16:0/2397292954 wait complete. 2024-08-17T02:45:27.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f4396000640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f43880ddd90 0x7f43880cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.012+0000 7f4396000640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4388053020 0x7f43880f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.013+0000 7f4396000640 1 --2- 172.21.15.16:0/2114570094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f438800c040 0x7f43880340c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.013+0000 7f4396000640 1 -- 172.21.15.16:0/2114570094 >> 172.21.15.16:0/2114570094 conn(0x7f4388009fa0 msgr2=0x7f4388039940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:27.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.013+0000 7f4396000640 1 -- 172.21.15.16:0/2114570094 shutdown_connections 2024-08-17T02:45:27.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.013+0000 7f4396000640 1 -- 172.21.15.16:0/2114570094 wait complete. 2024-08-17T02:45:27.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.020+0000 7f9348c00640 1 -- 172.21.15.16:0/4271043779 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f934415f1c0 con 0x7f9344069ed0 2024-08-17T02:45:27.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.023+0000 7f754f400640 1 -- 172.21.15.16:0/829799436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f754809c400 con 0x7f7548164e40 2024-08-17T02:45:27.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.023+0000 7f9339600640 1 -- 172.21.15.16:0/4271043779 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f934415f1c0 con 0x7f9344069ed0 2024-08-17T02:45:27.025 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2024-08-17T02:45:27.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.025+0000 7f753fe00640 1 -- 172.21.15.16:0/829799436 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f75340533a0 con 0x7f7548164e40 2024-08-17T02:45:27.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.025+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcd700c9c60 con 0x7fcd7006a410 2024-08-17T02:45:27.027 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2024-08-17T02:45:27.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.025+0000 7fcd65600640 1 -- 172.21.15.16:0/2073881829 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcd68074c50 con 0x7fcd7006a410 2024-08-17T02:45:27.028 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2024-08-17T02:45:27.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.027+0000 7f6f95e00640 1 -- 172.21.15.16:0/1646876883 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6f900cbaf0 con 0x7f6f900cd700 2024-08-17T02:45:27.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.028+0000 7f9348c00640 1 -- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9320065150 msgr2=0x7f9320067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.028+0000 7f9348c00640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9320065150 0x7f9320067600 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9344168490 tx=0x7f932c000bc0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.028+0000 7f9348c00640 1 -- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9344069ed0 msgr2=0x7f934415af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.028+0000 7f9348c00640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9344069ed0 0x7f934415af20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9334031000 tx=0x7f9334032000 comp rx=0 tx=0).stop 2024-08-17T02:45:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.028+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff4780c7f30 con 0x7ff4780d08d0 2024-08-17T02:45:27.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.028+0000 7f6f83e00640 1 -- 172.21.15.16:0/1646876883 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6f7c042020 con 0x7f6f900cd700 2024-08-17T02:45:27.031 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:27.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcd50065220 msgr2=0x7fcd500676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7fcd75200640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcd50065220 0x7fcd500676d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcd701529f0 tx=0x7fcd600091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd7006a410 msgr2=0x7fcd7014b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7fcd75200640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd7006a410 0x7fcd7014b760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcd680099f0 tx=0x7fcd68037000 comp rx=0 tx=0).stop 2024-08-17T02:45:27.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 shutdown_connections 2024-08-17T02:45:27.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7fcd75200640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcd50065220 0x7fcd500676d0 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcd701529f0 tx=0x7fcd600091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7fcd75200640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd701554f0 0x7fcd700ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7fcd75200640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd700b0010 0x7fcd70154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7fcd75200640 1 --2- 172.21.15.16:0/2073881829 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcd7006a410 0x7fcd7014b760 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 >> 172.21.15.16:0/2073881829 conn(0x7fcd700cc480 msgr2=0x7fcd700ca460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.029+0000 7f754f400640 1 -- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7528065150 msgr2=0x7f7528067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.029+0000 7f754f400640 1 --2- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7528065150 0x7f7528067600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f753803d000 tx=0x7f753803b000 comp rx=0 tx=0).stop 2024-08-17T02:45:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.029+0000 7f754f400640 1 -- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548164e40 msgr2=0x7f7548099d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.029+0000 7f754f400640 1 --2- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548164e40 0x7f7548099d30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f753400e470 tx=0x7f7534009040 comp rx=0 tx=0).stop 2024-08-17T02:45:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7f754f400640 1 -- 172.21.15.16:0/829799436 shutdown_connections 2024-08-17T02:45:27.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7f754f400640 1 --2- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7528065150 0x7f7528067600 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f753803d000 tx=0x7f753803b000 comp rx=0 tx=0).stop 2024-08-17T02:45:27.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7f754f400640 1 --2- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f754809a270 0x7f7548153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7f754f400640 1 --2- 172.21.15.16:0/829799436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548164e40 0x7f7548099d30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7f754f400640 1 --2- 172.21.15.16:0/829799436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f752c003af0 0x7f754809ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7f754f400640 1 -- 172.21.15.16:0/829799436 >> 172.21.15.16:0/829799436 conn(0x7f75480cfdf0 msgr2=0x7f7548067510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.029+0000 7f9348c00640 1 -- 172.21.15.16:0/4271043779 shutdown_connections 2024-08-17T02:45:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.029+0000 7f9348c00640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9320065150 0x7f9320067600 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.029+0000 7f9348c00640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9344165d20 0x7f9344163b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.029+0000 7f9348c00640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93440b1710 0x7f9344163610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.029+0000 7f9348c00640 1 --2- 172.21.15.16:0/4271043779 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9344069ed0 0x7f934415af20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.029+0000 7f9348c00640 1 -- 172.21.15.16:0/4271043779 >> 172.21.15.16:0/4271043779 conn(0x7f93440cc480 msgr2=0x7f9344166e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:27.035 INFO:teuthology.orchestra.run.smithi016.stdout:55834574851 2024-08-17T02:45:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.029+0000 7ff46ea00640 1 -- 172.21.15.16:0/3357845269 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff4600739b0 con 0x7ff4780d08d0 2024-08-17T02:45:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.030+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 shutdown_connections 2024-08-17T02:45:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.031+0000 7fcd75200640 1 -- 172.21.15.16:0/2073881829 wait complete. 2024-08-17T02:45:27.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.031+0000 7f9348c00640 1 -- 172.21.15.16:0/4271043779 shutdown_connections 2024-08-17T02:45:27.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.031+0000 7f9348c00640 1 -- 172.21.15.16:0/4271043779 wait complete. 2024-08-17T02:45:27.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.031+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff454065220 msgr2=0x7ff4540676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.032+0000 7ff47e600640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff454065220 0x7ff4540676d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff468007fd0 tx=0x7ff468007b00 comp rx=0 tx=0).stop 2024-08-17T02:45:27.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.032+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4780d08d0 msgr2=0x7ff478156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.032+0000 7ff47e600640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4780d08d0 0x7ff478156e80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4600098b0 tx=0x7ff460032000 comp rx=0 tx=0).stop 2024-08-17T02:45:27.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.032+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 shutdown_connections 2024-08-17T02:45:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.032+0000 7ff47e600640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff454065220 0x7ff4540676d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.033+0000 7ff47e600640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff478158ce0 0x7ff4781573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.033+0000 7ff47e600640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4780d08d0 0x7ff478156e80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.033+0000 7ff47e600640 1 --2- 172.21.15.16:0/3357845269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4780cb340 0x7ff478141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.033+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 >> 172.21.15.16:0/3357845269 conn(0x7ff4780cfdc0 msgr2=0x7ff4780c57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.033+0000 7f754f400640 1 -- 172.21.15.16:0/829799436 shutdown_connections 2024-08-17T02:45:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.033+0000 7f754f400640 1 -- 172.21.15.16:0/829799436 wait complete. 2024-08-17T02:45:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.033+0000 7f6f95e00640 1 -- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6f70065150 msgr2=0x7f6f70067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.033+0000 7f6f95e00640 1 --2- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6f70065150 0x7f6f70067600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f84007fd0 tx=0x7f6f840078d0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.033+0000 7f6f95e00640 1 -- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f900cd700 msgr2=0x7f6f90168de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.033+0000 7f6f95e00640 1 --2- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f900cd700 0x7f6f90168de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c035040 tx=0x7f6f7c032000 comp rx=0 tx=0).stop 2024-08-17T02:45:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.033+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 shutdown_connections 2024-08-17T02:45:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.033+0000 7ff47e600640 1 -- 172.21.15.16:0/3357845269 wait complete. 2024-08-17T02:45:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.034+0000 7f6f95e00640 1 -- 172.21.15.16:0/1646876883 shutdown_connections 2024-08-17T02:45:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.034+0000 7f6f95e00640 1 --2- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6f70065150 0x7f6f70067600 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.034+0000 7f6f95e00640 1 --2- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f90158c80 0x7f6f90169320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.034+0000 7f6f95e00640 1 --2- 172.21.15.16:0/1646876883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f900cd700 0x7f6f90168de0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.034+0000 7f6f95e00640 1 --2- 172.21.15.16:0/1646876883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f900cc9d0 0x7f6f900c3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.034+0000 7f6f95e00640 1 -- 172.21.15.16:0/1646876883 >> 172.21.15.16:0/1646876883 conn(0x7f6f90098e30 msgr2=0x7f6f900c8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:27.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.035+0000 7f6f95e00640 1 -- 172.21.15.16:0/1646876883 shutdown_connections 2024-08-17T02:45:27.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2024-08-17T02:45:27.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:27.035+0000 7f6f95e00640 1 -- 172.21.15.16:0/1646876883 wait complete. 2024-08-17T02:45:27.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2024-08-17T02:45:27.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-08-17T02:45:27.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2024-08-17T02:45:27.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2024-08-17T02:45:27.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.6 2024-08-17T02:45:27.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.7 2024-08-17T02:45:27.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2024-08-17T02:45:28.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-08-17T02:45:28.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-08-17T02:45:28.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-08-17T02:45:28.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-08-17T02:45:28.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-08-17T02:45:28.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-08-17T02:45:28.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-08-17T02:45:28.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-08-17T02:45:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.289+0000 7f17f6a00640 1 Processor -- start 2024-08-17T02:45:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.289+0000 7f17f6a00640 1 -- start start 2024-08-17T02:45:28.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.295+0000 7f17f6a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f17e800da90 0x7f17e800de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.295+0000 7f17f6a00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e8054970 con 0x7f17e80e01d0 2024-08-17T02:45:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.295+0000 7f17f6a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e8055210 con 0x7f17e80358c0 2024-08-17T02:45:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.295+0000 7f17f6a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e80362a0 con 0x7f17e800da90 2024-08-17T02:45:28.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.295+0000 7f4898c00640 1 Processor -- start 2024-08-17T02:45:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.295+0000 7f17efe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f17e800da90 0x7f17e800de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.296+0000 7f17f4a00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f17e80e01d0 0x7f17e80556a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46180/0 (socket says 172.21.15.16:46180) 2024-08-17T02:45:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.296+0000 7f17ef400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f17e80358c0 0x7f17e80dfac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:34994/0 (socket says 172.21.15.16:34994) 2024-08-17T02:45:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.296+0000 7f17f4a00640 1 -- 172.21.15.16:0/2521138107 learned_addr learned my addr 172.21.15.16:0/2521138107 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.297+0000 7f17efe00640 1 -- 172.21.15.16:0/2521138107 >> v1:172.21.15.142:6789/0 conn(0x7f17e80358c0 legacy=0x7f17e80dfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.297+0000 7f17efe00640 1 -- 172.21.15.16:0/2521138107 >> v1:172.21.15.16:6789/0 conn(0x7f17e80e01d0 legacy=0x7f17e80556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.297+0000 7f17efe00640 1 -- 172.21.15.16:0/2521138107 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e8054bc0 con 0x7f17e800da90 2024-08-17T02:45:28.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.297+0000 7f17eea00640 1 -- 172.21.15.16:0/2521138107 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327975784 0 0) 0x7f17e8054970 con 0x7f17e80e01d0 2024-08-17T02:45:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.298+0000 7f17ef400640 1 --1- 172.21.15.16:0/2521138107 >> v1:172.21.15.142:6789/0 conn(0x7f17e80358c0 0x7f17e80dfac0 :-1 s=CLOSED pgs=0 cs=0 l=0).handle_server_banner_and_identify state changed while learned_addr, mark_down or replacing must be happened just now 2024-08-17T02:45:28.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.296+0000 7f4898c00640 1 -- start start 2024-08-17T02:45:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.298+0000 7f1bc7e00640 1 Processor -- start 2024-08-17T02:45:28.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.298+0000 7f1bc7e00640 1 -- start start 2024-08-17T02:45:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.299+0000 7f17efe00640 1 --2- 172.21.15.16:0/2521138107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17e800da90 0x7f17e800de60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009850 tx=0x7f17e40090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=960e20431a9a01d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.299+0000 7f4898c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894152960 0x7f48941565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.299+0000 7f17eea00640 1 -- 172.21.15.16:0/2521138107 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e402ac70 con 0x7f17e800da90 2024-08-17T02:45:28.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.299+0000 7f17eea00640 1 -- 172.21.15.16:0/2521138107 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e402add0 con 0x7f17e800da90 2024-08-17T02:45:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.299+0000 7f17eea00640 1 -- 172.21.15.16:0/2521138107 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e4032670 con 0x7f17e800da90 2024-08-17T02:45:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.299+0000 7f4898c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f48940af7c0 0x7f48940afc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.299+0000 7f4898c00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4894159900 con 0x7f4894156130 2024-08-17T02:45:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.299+0000 7f4898c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48941590e0 con 0x7f48940af7c0 2024-08-17T02:45:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.299+0000 7f4898c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b0140 con 0x7f4894152960 2024-08-17T02:45:28.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.301+0000 7f17f6a00640 1 -- 172.21.15.16:0/2521138107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17e800da90 msgr2=0x7f17e800de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.301+0000 7f1bc7e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1bc8152960 0x7f1bc81565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.301+0000 7f17f6a00640 1 --2- 172.21.15.16:0/2521138107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17e800da90 0x7f17e800de60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009850 tx=0x7f17e40090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.301+0000 7f17f6a00640 1 -- 172.21.15.16:0/2521138107 shutdown_connections 2024-08-17T02:45:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.301+0000 7f17f6a00640 1 --2- 172.21.15.16:0/2521138107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17e800da90 0x7f17e800de60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.301+0000 7f17f6a00640 1 -- 172.21.15.16:0/2521138107 >> 172.21.15.16:0/2521138107 conn(0x7f17e8057870 msgr2=0x7f17e8009ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.301+0000 7f1bc6a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1bc8152960 0x7f1bc81565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.301+0000 7f4893400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894152960 0x7f48941565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.301+0000 7f4893400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894152960 0x7f48941565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:53094/0 (socket says 172.21.15.16:53094) 2024-08-17T02:45:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.301+0000 7f4893400640 1 -- 172.21.15.16:0/3207709206 learned_addr learned my addr 172.21.15.16:0/3207709206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.301+0000 7f17f6a00640 1 -- 172.21.15.16:0/2521138107 shutdown_connections 2024-08-17T02:45:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.302+0000 7f17f6a00640 1 -- 172.21.15.16:0/2521138107 wait complete. 2024-08-17T02:45:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.302+0000 7f1bc7e00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc8159900 con 0x7f1bc8156130 2024-08-17T02:45:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.302+0000 7f1bc7400640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f1bc80af7c0 0x7f1bc80afbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:35002/0 (socket says 172.21.15.16:35002) 2024-08-17T02:45:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.302+0000 7f1bc7400640 1 -- 172.21.15.16:0/362163408 learned_addr learned my addr 172.21.15.16:0/362163408 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.302+0000 7f1bc7e00640 1 -- 172.21.15.16:0/362163408 --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc81590e0 con 0x7f1bc80af7c0 2024-08-17T02:45:28.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.302+0000 7f1bc7e00640 1 -- 172.21.15.16:0/362163408 --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc8159340 con 0x7f1bc8152960 2024-08-17T02:45:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.301+0000 7f4893e00640 1 --2- 172.21.15.16:0/3207709206 >> v2:172.21.15.142:3300/0 conn(0x7f48940af7c0 0x7f48940afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.302+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080198710 0 0) 0x7f1bc81590e0 con 0x7f1bc80af7c0 2024-08-17T02:45:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.302+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bac0034e0 con 0x7f1bc80af7c0 2024-08-17T02:45:28.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.303+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 <== mon.1 v1:172.21.15.142:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3307784971 0 0) 0x7f1bac0034e0 con 0x7f1bc80af7c0 2024-08-17T02:45:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.303+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 >> v2:172.21.15.16:3300/0 conn(0x7f1bc8152960 msgr2=0x7f1bc81565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.303+0000 7f1bc5600640 1 --2- 172.21.15.16:0/362163408 >> v2:172.21.15.16:3300/0 conn(0x7f1bc8152960 0x7f1bc81565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.303+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 >> v1:172.21.15.16:6789/0 conn(0x7f1bc8156130 legacy=0x7f1bc81589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.303+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 --> v1:172.21.15.142:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc8159620 con 0x7f1bc80af7c0 2024-08-17T02:45:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.303+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 <== mon.1 v1:172.21.15.142:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7f1bc00050f0 con 0x7f1bc80af7c0 2024-08-17T02:45:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.304+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 >> v1:172.21.15.142:6789/0 conn(0x7f1bc80af7c0 legacy=0x7f1bc80afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.304+0000 7f4893400640 1 -- 172.21.15.16:0/3207709206 >> v2:172.21.15.142:3300/0 conn(0x7f48940af7c0 msgr2=0x7f48940afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.304+0000 7f1bc5600640 1 --2- 172.21.15.16:0/362163408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bac003aa0 0x7f1bac003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.304+0000 7f1bc5600640 1 --2- 172.21.15.16:0/362163408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bac004780 0x7f1bac004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.304+0000 7f1bc5600640 1 --2- 172.21.15.16:0/362163408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 0x7f1bac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.304+0000 7f17f6a00640 1 Processor -- start 2024-08-17T02:45:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.305+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc8159340 con 0x7f1bac003aa0 2024-08-17T02:45:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.305+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac00c100 con 0x7f1bac005250 2024-08-17T02:45:28.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.305+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac00c2c0 con 0x7f1bac004780 2024-08-17T02:45:28.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.304+0000 7f4892a00640 1 -- 172.21.15.16:0/3207709206 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680160755 0 0) 0x7f4894159900 con 0x7f4894156130 2024-08-17T02:45:28.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.304+0000 7f4893400640 1 --2- 172.21.15.16:0/3207709206 >> v2:172.21.15.142:3300/0 conn(0x7f48940af7c0 0x7f48940afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.304+0000 7f4893400640 1 -- 172.21.15.16:0/3207709206 >> v1:172.21.15.16:6789/0 conn(0x7f4894156130 legacy=0x7f48941589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.304+0000 7f4893400640 1 -- 172.21.15.16:0/3207709206 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4894159340 con 0x7f4894152960 2024-08-17T02:45:28.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.305+0000 7f1bc6a00640 1 --2- 172.21.15.16:0/362163408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bac003aa0 0x7f1bac003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.305+0000 7f17f6a00640 1 -- start start 2024-08-17T02:45:28.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.305+0000 7f1bc6000640 1 --2- 172.21.15.16:0/362163408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bac004780 0x7f1bac004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.305+0000 7f4893e00640 1 --2- 172.21.15.16:0/3207709206 >> v2:172.21.15.142:3300/0 conn(0x7f48940af7c0 0x7f48940afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.305+0000 7f1bc7400640 1 --2- 172.21.15.16:0/362163408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 0x7f1bac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.306+0000 7f1bc7400640 1 -- 172.21.15.16:0/362163408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bac004780 msgr2=0x7f1bac004bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.306+0000 7f1bc7400640 1 --2- 172.21.15.16:0/362163408 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bac004780 0x7f1bac004bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.306+0000 7f1bc7400640 1 -- 172.21.15.16:0/362163408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bac003aa0 msgr2=0x7f1bac003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.306+0000 7f1bc7400640 1 --2- 172.21.15.16:0/362163408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bac003aa0 0x7f1bac003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.306+0000 7f1bc7400640 1 -- 172.21.15.16:0/362163408 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac0034e0 con 0x7f1bac005250 2024-08-17T02:45:28.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.307+0000 7f4893400640 1 --2- 172.21.15.16:0/3207709206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894152960 0x7f48941565a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f488400d1a0 tx=0x7f4884007690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a730ec0e5cf5329 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.307+0000 7f17f6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17e800da90 0x7f17e80e30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.307+0000 7f17f6a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e80358c0 0x7f17e804fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.307+0000 7f17f6a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17e80e01d0 0x7f17e80ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.307+0000 7f1bc7400640 1 -- 172.21.15.16:0/362163408 reap_dead start 2024-08-17T02:45:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.308+0000 7f4892a00640 1 -- 172.21.15.16:0/3207709206 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884033650 con 0x7f4894152960 2024-08-17T02:45:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.308+0000 7f17f6a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e80d30e0 con 0x7f17e80358c0 2024-08-17T02:45:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.308+0000 7f17f4a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17e80e01d0 0x7f17e80ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.308+0000 7f17f4a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17e80e01d0 0x7f17e80ee3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:53114/0 (socket says 172.21.15.16:53114) 2024-08-17T02:45:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.308+0000 7f17f4a00640 1 -- 172.21.15.16:0/652472802 learned_addr learned my addr 172.21.15.16:0/652472802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.308+0000 7f1bc7400640 1 --2- 172.21.15.16:0/362163408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 0x7f1bac00b9d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00016f0 tx=0x7f1bc0008850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1545d1abe079f731 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.308+0000 7f4892a00640 1 -- 172.21.15.16:0/3207709206 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48840337b0 con 0x7f4894152960 2024-08-17T02:45:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.308+0000 7f4892a00640 1 -- 172.21.15.16:0/3207709206 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884033b90 con 0x7f4894152960 2024-08-17T02:45:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.308+0000 7f17f6a00640 1 -- 172.21.15.16:0/652472802 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e80ef610 con 0x7f17e800da90 2024-08-17T02:45:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.308+0000 7f17f6a00640 1 -- 172.21.15.16:0/652472802 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e8050390 con 0x7f17e80e01d0 2024-08-17T02:45:28.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.308+0000 7f17efe00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17e800da90 0x7f17e80e30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.309+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0008150 con 0x7f1bac005250 2024-08-17T02:45:28.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.309+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc00082b0 con 0x7f1bac005250 2024-08-17T02:45:28.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.309+0000 7f4898c00640 1 -- 172.21.15.16:0/3207709206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894152960 msgr2=0x7f48941565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.309+0000 7f4898c00640 1 --2- 172.21.15.16:0/3207709206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894152960 0x7f48941565a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f488400d1a0 tx=0x7f4884007690 comp rx=0 tx=0).stop 2024-08-17T02:45:28.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.309+0000 7f17f4a00640 1 -- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17e800da90 msgr2=0x7f17e80e30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.309+0000 7f17f4a00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17e800da90 0x7f17e80e30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.309+0000 7f17f4a00640 1 -- 172.21.15.16:0/652472802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e80358c0 msgr2=0x7f17e804fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.310+0000 7f4898c00640 1 -- 172.21.15.16:0/3207709206 shutdown_connections 2024-08-17T02:45:28.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.309+0000 7f17ef400640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e80358c0 0x7f17e804fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.310+0000 7f4898c00640 1 --2- 172.21.15.16:0/3207709206 >> v2:172.21.15.142:3300/0 conn(0x7f48940af7c0 0x7f48940afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.310+0000 7f4898c00640 1 --2- 172.21.15.16:0/3207709206 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894152960 0x7f48941565a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.310+0000 7f4898c00640 1 -- 172.21.15.16:0/3207709206 >> 172.21.15.16:0/3207709206 conn(0x7f48940cb790 msgr2=0x7f48940cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.310+0000 7f17f4a00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e80358c0 0x7f17e804fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.310+0000 7f1bc5600640 1 -- 172.21.15.16:0/362163408 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0008150 con 0x7f1bac005250 2024-08-17T02:45:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.310+0000 7f17f4a00640 1 -- 172.21.15.16:0/652472802 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e4009390 con 0x7f17e80e01d0 2024-08-17T02:45:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.310+0000 7f17ef400640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e80358c0 0x7f17e804fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.310+0000 7f17efe00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17e800da90 0x7f17e80e30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:28.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.311+0000 7f1bc7e00640 1 -- 172.21.15.16:0/362163408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 msgr2=0x7f1bac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.310+0000 7f4898c00640 1 -- 172.21.15.16:0/3207709206 shutdown_connections 2024-08-17T02:45:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.310+0000 7f4898c00640 1 -- 172.21.15.16:0/3207709206 wait complete. 2024-08-17T02:45:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.311+0000 7f1bc7e00640 1 --2- 172.21.15.16:0/362163408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 0x7f1bac00b9d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00016f0 tx=0x7f1bc0008850 comp rx=0 tx=0).stop 2024-08-17T02:45:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.311+0000 7f17f4a00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17e80e01d0 0x7f17e80ee3e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00c5b0 tx=0x7f17dc007d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.311+0000 7f1bc7e00640 1 -- 172.21.15.16:0/362163408 shutdown_connections 2024-08-17T02:45:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.311+0000 7f1bc7e00640 1 --2- 172.21.15.16:0/362163408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 0x7f1bac00b9d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.311+0000 7f1bc7e00640 1 -- 172.21.15.16:0/362163408 >> 172.21.15.16:0/362163408 conn(0x7f1bc80cb790 msgr2=0x7f1bc80cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.312+0000 7f1bc7e00640 1 -- 172.21.15.16:0/362163408 shutdown_connections 2024-08-17T02:45:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.312+0000 7f17e3400640 1 -- 172.21.15.16:0/652472802 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc005aa0 con 0x7f17e80e01d0 2024-08-17T02:45:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.312+0000 7f17e3400640 1 -- 172.21.15.16:0/652472802 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc005c00 con 0x7f17e80e01d0 2024-08-17T02:45:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.312+0000 7f17e3400640 1 -- 172.21.15.16:0/652472802 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0070f0 con 0x7f17e80e01d0 2024-08-17T02:45:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.312+0000 7f1bc7e00640 1 -- 172.21.15.16:0/362163408 wait complete. 2024-08-17T02:45:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.312+0000 7f4898c00640 1 Processor -- start 2024-08-17T02:45:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.313+0000 7f4898c00640 1 -- start start 2024-08-17T02:45:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.312+0000 7f17f6a00640 1 -- 172.21.15.16:0/652472802 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e8050660 con 0x7f17e80e01d0 2024-08-17T02:45:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.312+0000 7f17f6a00640 1 -- 172.21.15.16:0/652472802 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e8036f30 con 0x7f17e80e01d0 2024-08-17T02:45:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.315+0000 7f4898c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48940af7c0 0x7f4894150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.315+0000 7f1bc7e00640 1 Processor -- start 2024-08-17T02:45:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.315+0000 7f4898c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894152960 0x7f4894153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.315+0000 7f1bc7e00640 1 -- start start 2024-08-17T02:45:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.315+0000 7f4898c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4894156130 0x7f4894154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.316+0000 7f4898c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48941513d0 con 0x7f48940af7c0 2024-08-17T02:45:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.316+0000 7f4898c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489416d930 con 0x7f4894156130 2024-08-17T02:45:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.316+0000 7f4898c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489416dc10 con 0x7f4894152960 2024-08-17T02:45:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.317+0000 7f4893e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4894156130 0x7f4894154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.317+0000 7f4893e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4894156130 0x7f4894154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36842/0 (socket says 172.21.15.16:36842) 2024-08-17T02:45:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.317+0000 7f1bc7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 0x7f1bc8149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.317+0000 7f4893e00640 1 -- 172.21.15.16:0/1082119407 learned_addr learned my addr 172.21.15.16:0/1082119407 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.318+0000 7f4893400640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48940af7c0 0x7f4894150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.318+0000 7f17f6a00640 1 -- 172.21.15.16:0/652472802 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17cc005210 con 0x7f17e80e01d0 2024-08-17T02:45:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.317+0000 7f1bc7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc814e1d0 0x7f1bc81690a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.318+0000 7f1bc7e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bc806d8b0 0x7f1bc813ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.318+0000 7f488aa00640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894152960 0x7f4894153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.318+0000 7f1bc6000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc814e1d0 0x7f1bc81690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.318+0000 7f1bc7400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bc806d8b0 0x7f1bc813ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.319+0000 7f1bc7e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc81695e0 con 0x7f1bc814e1d0 2024-08-17T02:45:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.318+0000 7f1bc6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 0x7f1bc8149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.319+0000 7f1bc6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 0x7f1bc8149930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36848/0 (socket says 172.21.15.16:36848) 2024-08-17T02:45:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.319+0000 7f1bc6a00640 1 -- 172.21.15.16:0/167247389 learned_addr learned my addr 172.21.15.16:0/167247389 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.319+0000 7f4893e00640 1 -- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894152960 msgr2=0x7f4894153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.319+0000 7f4893e00640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894152960 0x7f4894153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.319+0000 7f4893e00640 1 -- 172.21.15.16:0/1082119407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48940af7c0 msgr2=0x7f4894150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.319+0000 7f4893e00640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48940af7c0 0x7f4894150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.319+0000 7f4893e00640 1 -- 172.21.15.16:0/1082119407 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488400be70 con 0x7f4894156130 2024-08-17T02:45:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.318+0000 7f17e3400640 1 -- 172.21.15.16:0/652472802 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f17dc007520 con 0x7f17e80e01d0 2024-08-17T02:45:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.319+0000 7f17e3400640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f17d00652a0 0x7f17d0067750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.320+0000 7f17e3400640 1 -- 172.21.15.16:0/652472802 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f17dc081e00 con 0x7f17e80e01d0 2024-08-17T02:45:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.319+0000 7f1bc7e00640 1 -- 172.21.15.16:0/167247389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc814d450 con 0x7f1bac005250 2024-08-17T02:45:28.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.319+0000 7f1bc7e00640 1 -- 172.21.15.16:0/167247389 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc814d6f0 con 0x7f1bc806d8b0 2024-08-17T02:45:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.320+0000 7f4893e00640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4894156130 0x7f4894154240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f488c00d890 tx=0x7f488c008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.320+0000 7f1bc6a00640 1 -- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bc806d8b0 msgr2=0x7f1bc813ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.320+0000 7f1bc6a00640 1 --2- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bc806d8b0 0x7f1bc813ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.320+0000 7f1bc6a00640 1 -- 172.21.15.16:0/167247389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc814e1d0 msgr2=0x7f1bc81690a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.320+0000 7f1bc6a00640 1 --2- 172.21.15.16:0/167247389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc814e1d0 0x7f1bc81690a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.320+0000 7f1bc6a00640 1 -- 172.21.15.16:0/167247389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc814d9d0 con 0x7f1bac005250 2024-08-17T02:45:28.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.320+0000 7f17efe00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f17d00652a0 0x7f17d0067750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.321+0000 7f488b400640 1 -- 172.21.15.16:0/1082119407 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c00de20 con 0x7f4894156130 2024-08-17T02:45:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.321+0000 7f4898c00640 1 -- 172.21.15.16:0/1082119407 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489416def0 con 0x7f4894156130 2024-08-17T02:45:28.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.321+0000 7f1bc6a00640 1 --2- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 0x7f1bc8149930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0011a60 tx=0x7f1bb00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.321+0000 7f488b400640 1 -- 172.21.15.16:0/1082119407 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c0098b0 con 0x7f4894156130 2024-08-17T02:45:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.321+0000 7f4898c00640 1 -- 172.21.15.16:0/1082119407 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489416e4b0 con 0x7f4894156130 2024-08-17T02:45:28.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.322+0000 7f488b400640 1 -- 172.21.15.16:0/1082119407 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c00de20 con 0x7f4894156130 2024-08-17T02:45:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.323+0000 7f1bbe000640 1 -- 172.21.15.16:0/167247389 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb000fb00 con 0x7f1bac005250 2024-08-17T02:45:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.323+0000 7f1bc7e00640 1 -- 172.21.15.16:0/167247389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc814dcb0 con 0x7f1bac005250 2024-08-17T02:45:28.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.323+0000 7f1bc7e00640 1 -- 172.21.15.16:0/167247389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc8149f30 con 0x7f1bac005250 2024-08-17T02:45:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.323+0000 7f1bbe000640 1 -- 172.21.15.16:0/167247389 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb000fc60 con 0x7f1bac005250 2024-08-17T02:45:28.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.324+0000 7f4898c00640 1 -- 172.21.15.16:0/1082119407 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f489416e6c0 con 0x7f4894156130 2024-08-17T02:45:28.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.324+0000 7f1bbe000640 1 -- 172.21.15.16:0/167247389 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb000fb00 con 0x7f1bac005250 2024-08-17T02:45:28.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.324+0000 7f17efe00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f17d00652a0 0x7f17d0067750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f17e4034000 tx=0x7f17e4032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.325+0000 7f488b400640 1 -- 172.21.15.16:0/1082119407 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f488c009cb0 con 0x7f4894156130 2024-08-17T02:45:28.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.325+0000 7f1bc7e00640 1 -- 172.21.15.16:0/167247389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba0005210 con 0x7f1bac005250 2024-08-17T02:45:28.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.326+0000 7f17e3400640 1 -- 172.21.15.16:0/652472802 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f17dc04ff20 con 0x7f17e80e01d0 2024-08-17T02:45:28.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.326+0000 7f488b400640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4870065230 0x7f48700676e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.326+0000 7f488b400640 1 -- 172.21.15.16:0/1082119407 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f488c0824c0 con 0x7f4894156130 2024-08-17T02:45:28.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.328+0000 7f4893400640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4870065230 0x7f48700676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.328+0000 7f1bbe000640 1 -- 172.21.15.16:0/167247389 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1bb0014650 con 0x7f1bac005250 2024-08-17T02:45:28.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.329+0000 7f4893400640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4870065230 0x7f48700676e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4884038000 tx=0x7f488403b000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.330+0000 7f488b400640 1 -- 172.21.15.16:0/1082119407 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f488c0504e0 con 0x7f4894156130 2024-08-17T02:45:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.331+0000 7f1bbe000640 1 --2- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1ba4065170 0x7f1ba4067620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.332+0000 7f1bbe000640 1 -- 172.21.15.16:0/167247389 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1bb0018030 con 0x7f1bac005250 2024-08-17T02:45:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.332+0000 7f1bc6000640 1 --2- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1ba4065170 0x7f1ba4067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.334+0000 7f1bbe000640 1 -- 172.21.15.16:0/167247389 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1bb0050b80 con 0x7f1bac005250 2024-08-17T02:45:28.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.335+0000 7f1bc6000640 1 --2- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1ba4065170 0x7f1ba4067620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8015fd0 tx=0x7f1bb8017040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.375+0000 7fcedfe00640 1 Processor -- start 2024-08-17T02:45:28.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.376+0000 7fcedfe00640 1 -- start start 2024-08-17T02:45:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.378+0000 7fcedfe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcee00afdd0 0x7fcee01563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.378+0000 7fcedfe00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee0157690 con 0x7fcee00b0270 2024-08-17T02:45:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.378+0000 7fcedfe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee0156a80 con 0x7fcee0155900 2024-08-17T02:45:28.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.378+0000 7fcedfe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee0156d60 con 0x7fcee00afdd0 2024-08-17T02:45:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.378+0000 7fcedea00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fcee0155900 0x7fcee0155cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:35010/0 (socket says 172.21.15.16:35010) 2024-08-17T02:45:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.378+0000 7fcedea00640 1 -- 172.21.15.16:0/1165008923 learned_addr learned my addr 172.21.15.16:0/1165008923 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.379+0000 7fcedf400640 1 --2- 172.21.15.16:0/1165008923 >> v2:172.21.15.142:3300/0 conn(0x7fcee00afdd0 0x7fcee01563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.379+0000 7f2c37800640 1 Processor -- start 2024-08-17T02:45:28.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.380+0000 7f2c37800640 1 -- start start 2024-08-17T02:45:28.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.379+0000 7fcedf400640 1 -- 172.21.15.16:0/1165008923 >> v1:172.21.15.142:6789/0 conn(0x7fcee0155900 legacy=0x7fcee0155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.379+0000 7fcedf400640 1 -- 172.21.15.16:0/1165008923 >> v1:172.21.15.16:6789/0 conn(0x7fcee00b0270 legacy=0x7fcee00ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.379+0000 7fcedf400640 1 -- 172.21.15.16:0/1165008923 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee0157040 con 0x7fcee00afdd0 2024-08-17T02:45:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.382+0000 7f2c37800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c280e0860 0x7f2c28051590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.382+0000 7f2c37800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0c30 0x7f2c28008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.382+0000 7fcedf400640 1 --2- 172.21.15.16:0/1165008923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee00afdd0 0x7fcee01563e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fced8009850 tx=0x7fced80090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7df0557f633abec8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.382+0000 7fcede000640 1 -- 172.21.15.16:0/1165008923 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced802ab00 con 0x7fcee00afdd0 2024-08-17T02:45:28.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.382+0000 7f2c37800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c28051100 con 0x7f2c280091c0 2024-08-17T02:45:28.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.382+0000 7f2c37800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c280502e0 con 0x7f2c280e0860 2024-08-17T02:45:28.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.382+0000 7f2c37800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c28009640 con 0x7f2c280e0c30 2024-08-17T02:45:28.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.382+0000 7fcede000640 1 -- 172.21.15.16:0/1165008923 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced802ac60 con 0x7fcee00afdd0 2024-08-17T02:45:28.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.382+0000 7fcede000640 1 -- 172.21.15.16:0/1165008923 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8032440 con 0x7fcee00afdd0 2024-08-17T02:45:28.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.383+0000 7f2c2fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0c30 0x7f2c28008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.383+0000 7fd901600640 1 Processor -- start 2024-08-17T02:45:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.383+0000 7f2c34e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c280e0860 0x7f2c28051590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.383+0000 7f2c2fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0c30 0x7f2c28008c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:53152/0 (socket says 172.21.15.16:53152) 2024-08-17T02:45:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.383+0000 7f2c2fe00640 1 -- 172.21.15.16:0/1840180724 learned_addr learned my addr 172.21.15.16:0/1840180724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.383+0000 7fcedfe00640 1 -- 172.21.15.16:0/1165008923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee00afdd0 msgr2=0x7fcee01563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.383+0000 7fcedfe00640 1 --2- 172.21.15.16:0/1165008923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee00afdd0 0x7fcee01563e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fced8009850 tx=0x7fced80090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.384+0000 7fd901600640 1 -- start start 2024-08-17T02:45:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.384+0000 7fcedfe00640 1 -- 172.21.15.16:0/1165008923 shutdown_connections 2024-08-17T02:45:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.384+0000 7fcedfe00640 1 --2- 172.21.15.16:0/1165008923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee00afdd0 0x7fcee01563e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.384+0000 7fcedfe00640 1 -- 172.21.15.16:0/1165008923 >> 172.21.15.16:0/1165008923 conn(0x7fcee0067750 msgr2=0x7fcee00cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.384+0000 7f2c2f400640 1 -- 172.21.15.16:0/1840180724 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784784569 0 0) 0x7f2c28051100 con 0x7f2c280091c0 2024-08-17T02:45:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.384+0000 7f2c2f400640 1 -- 172.21.15.16:0/1840180724 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c180034e0 con 0x7f2c280091c0 2024-08-17T02:45:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.384+0000 7f2c2fe00640 1 -- 172.21.15.16:0/1840180724 >> v2:172.21.15.16:3300/0 conn(0x7f2c280e0860 msgr2=0x7f2c28051590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:28.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.384+0000 7f2c2fe00640 1 --2- 172.21.15.16:0/1840180724 >> v2:172.21.15.16:3300/0 conn(0x7f2c280e0860 0x7f2c28051590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.384+0000 7f2c2fe00640 1 -- 172.21.15.16:0/1840180724 >> v1:172.21.15.142:6789/0 conn(0x7f2c280091c0 legacy=0x7f2c28006a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.384+0000 7f2c2fe00640 1 -- 172.21.15.16:0/1840180724 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c280504f0 con 0x7f2c280e0c30 2024-08-17T02:45:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.384+0000 7fcedfe00640 1 -- 172.21.15.16:0/1165008923 shutdown_connections 2024-08-17T02:45:28.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.386+0000 7f6a85200640 1 Processor -- start 2024-08-17T02:45:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.386+0000 7f2c2fe00640 1 --2- 172.21.15.16:0/1840180724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0c30 0x7f2c28008c80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009910 tx=0x7f2c240091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e1473e32971c2a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.386+0000 7f2c2f400640 1 -- 172.21.15.16:0/1840180724 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2402adb0 con 0x7f2c280e0c30 2024-08-17T02:45:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.386+0000 7f2c2f400640 1 -- 172.21.15.16:0/1840180724 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c24032440 con 0x7f2c280e0c30 2024-08-17T02:45:28.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.386+0000 7f2c2f400640 1 -- 172.21.15.16:0/1840180724 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24032820 con 0x7f2c280e0c30 2024-08-17T02:45:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.387+0000 7fd901600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8fc13f4a0 0x7fd8fc0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.387+0000 7fd901600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8fc06bf30 0x7fd8fc0ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.387+0000 7fd901600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc06baa0 con 0x7fd8fc06a8d0 2024-08-17T02:45:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.387+0000 7fd901600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8fc06ae90 con 0x7fd8fc06bf30 2024-08-17T02:45:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.387+0000 7fd901600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc06b170 con 0x7fd8fc13f4a0 2024-08-17T02:45:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7fd8fa000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8fc06bf30 0x7fd8fc0ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7fd8fa000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8fc06bf30 0x7fd8fc0ae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58200/0 (socket says 172.21.15.16:58200) 2024-08-17T02:45:28.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7fd8fa000640 1 -- 172.21.15.16:0/968584941 learned_addr learned my addr 172.21.15.16:0/968584941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.386+0000 7fcedfe00640 1 -- 172.21.15.16:0/1165008923 wait complete. 2024-08-17T02:45:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7f2c37800640 1 -- 172.21.15.16:0/1840180724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0c30 msgr2=0x7f2c28008c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7fcedfe00640 1 Processor -- start 2024-08-17T02:45:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.386+0000 7f6a85200640 1 -- start start 2024-08-17T02:45:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7f6a85200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 0x7f6a7804f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7f6a85200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6a7804ff20 0x7f6a78005f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7f6a85200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6a78006440 0x7f6a780ea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7f6a85200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a780375b0 con 0x7f6a78006440 2024-08-17T02:45:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7f6a85200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a78006c30 con 0x7f6a7804ff20 2024-08-17T02:45:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7f6a85200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a78006f10 con 0x7f6a780e09f0 2024-08-17T02:45:28.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7fd8fa000640 1 -- 172.21.15.16:0/968584941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8fc13f4a0 msgr2=0x7fd8fc0ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.388+0000 7fd8fa000640 1 --2- 172.21.15.16:0/968584941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8fc13f4a0 0x7fd8fc0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.389+0000 7fd8fa000640 1 -- 172.21.15.16:0/968584941 >> v1:172.21.15.142:6789/0 conn(0x7fd8fc06a8d0 legacy=0x7fd8fc066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.389+0000 7fd8fa000640 1 -- 172.21.15.16:0/968584941 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc06b450 con 0x7fd8fc06bf30 2024-08-17T02:45:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.389+0000 7fcedfe00640 1 -- start start 2024-08-17T02:45:28.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.389+0000 7f2c37800640 1 --2- 172.21.15.16:0/1840180724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0c30 0x7f2c28008c80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009910 tx=0x7f2c240091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.389+0000 7f6a7ea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6a7804ff20 0x7f6a78005f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.389+0000 7f6a7ea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6a7804ff20 0x7f6a78005f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36878/0 (socket says 172.21.15.16:36878) 2024-08-17T02:45:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.389+0000 7f6a7ea00640 1 -- 172.21.15.16:0/3203229992 learned_addr learned my addr 172.21.15.16:0/3203229992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.389+0000 7f2c37800640 1 -- 172.21.15.16:0/1840180724 shutdown_connections 2024-08-17T02:45:28.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.389+0000 7f2c37800640 1 --2- 172.21.15.16:0/1840180724 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0c30 0x7f2c28008c80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.390+0000 7f2c37800640 1 --2- 172.21.15.16:0/1840180724 >> v2:172.21.15.16:3300/0 conn(0x7f2c280e0860 0x7f2c28051590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.390+0000 7f2c37800640 1 -- 172.21.15.16:0/1840180724 >> 172.21.15.16:0/1840180724 conn(0x7f2c28052710 msgr2=0x7f2c28052b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.390+0000 7f6a7f400640 1 --2- 172.21.15.16:0/3203229992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 0x7f6a7804f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.390+0000 7f6a7ea00640 1 -- 172.21.15.16:0/3203229992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 msgr2=0x7f6a7804f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:28.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.390+0000 7f6a7fe00640 1 --2- 172.21.15.16:0/3203229992 >> v2:172.21.15.16:3300/0 conn(0x7f6a78006440 0x7f6a780ea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.390+0000 7fd8fa000640 1 --2- 172.21.15.16:0/968584941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8fc06bf30 0x7fd8fc0ae470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009b70 tx=0x7fd8ec0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1002f4e6242d939 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.390+0000 7f6a7ea00640 1 --2- 172.21.15.16:0/3203229992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 0x7f6a7804f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.390+0000 7f6a7ea00640 1 -- 172.21.15.16:0/3203229992 >> v2:172.21.15.16:3300/0 conn(0x7f6a78006440 msgr2=0x7f6a780ea010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.390+0000 7f6a7ea00640 1 --2- 172.21.15.16:0/3203229992 >> v2:172.21.15.16:3300/0 conn(0x7f6a78006440 0x7f6a780ea010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.391+0000 7f6a7ea00640 1 -- 172.21.15.16:0/3203229992 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a780071f0 con 0x7f6a7804ff20 2024-08-17T02:45:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.391+0000 7fcedfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee00afdd0 0x7fcee00a7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.391+0000 7fcedfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee00b0270 0x7fcee0167f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.391+0000 7fd8f9600640 1 -- 172.21.15.16:0/968584941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec031400 con 0x7fd8fc06bf30 2024-08-17T02:45:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.391+0000 7fd8f9600640 1 -- 172.21.15.16:0/968584941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec02d050 con 0x7fd8fc06bf30 2024-08-17T02:45:28.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.391+0000 7fd8f9600640 1 -- 172.21.15.16:0/968584941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec0317d0 con 0x7fd8fc06bf30 2024-08-17T02:45:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.391+0000 7fcedea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee00afdd0 0x7fcee00a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.391+0000 7fcedfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcee0155900 0x7fcee0148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.391+0000 7fcedea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee00afdd0 0x7fcee00a7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36894/0 (socket says 172.21.15.16:36894) 2024-08-17T02:45:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.392+0000 7fcedfe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0147ea0 con 0x7fcee00b0270 2024-08-17T02:45:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.392+0000 7fcedea00640 1 -- 172.21.15.16:0/669682288 learned_addr learned my addr 172.21.15.16:0/669682288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.392+0000 7fcedfe00640 1 -- 172.21.15.16:0/669682288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0148180 con 0x7fcee00afdd0 2024-08-17T02:45:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.392+0000 7fcedfe00640 1 -- 172.21.15.16:0/669682288 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0148460 con 0x7fcee0155900 2024-08-17T02:45:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.392+0000 7fcedf400640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcee0155900 0x7fcee0148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.392+0000 7f6a7f400640 1 --2- 172.21.15.16:0/3203229992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 0x7f6a7804f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.392+0000 7fced6000640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee00b0270 0x7fcee0167f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.392+0000 7f6a7fe00640 1 --2- 172.21.15.16:0/3203229992 >> v2:172.21.15.16:3300/0 conn(0x7f6a78006440 0x7f6a780ea010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.392+0000 7fd901600640 1 -- 172.21.15.16:0/968584941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8fc06bf30 msgr2=0x7fd8fc0ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.392+0000 7fd901600640 1 --2- 172.21.15.16:0/968584941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8fc06bf30 0x7fd8fc0ae470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009b70 tx=0x7fd8ec0091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fcedea00640 1 -- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcee0155900 msgr2=0x7fcee0148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fcedea00640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcee0155900 0x7fcee0148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fcedea00640 1 -- 172.21.15.16:0/669682288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee00b0270 msgr2=0x7fcee0167f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fd901600640 1 -- 172.21.15.16:0/968584941 shutdown_connections 2024-08-17T02:45:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fd901600640 1 --2- 172.21.15.16:0/968584941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8fc06bf30 0x7fd8fc0ae470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fd901600640 1 --2- 172.21.15.16:0/968584941 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8fc13f4a0 0x7fd8fc0ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fd901600640 1 -- 172.21.15.16:0/968584941 >> 172.21.15.16:0/968584941 conn(0x7fd8fc06d8f0 msgr2=0x7fd8fc06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fcedea00640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee00b0270 0x7fcee0167f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fcedea00640 1 -- 172.21.15.16:0/669682288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced8009390 con 0x7fcee00afdd0 2024-08-17T02:45:28.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7f2c37800640 1 -- 172.21.15.16:0/1840180724 shutdown_connections 2024-08-17T02:45:28.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fced6000640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee00b0270 0x7fcee0167f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:28.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fcedf400640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcee0155900 0x7fcee0148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:28.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fd901600640 1 -- 172.21.15.16:0/968584941 shutdown_connections 2024-08-17T02:45:28.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7fd901600640 1 -- 172.21.15.16:0/968584941 wait complete. 2024-08-17T02:45:28.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.393+0000 7f2c37800640 1 -- 172.21.15.16:0/1840180724 wait complete. 2024-08-17T02:45:28.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.394+0000 7f6a7ea00640 1 --2- 172.21.15.16:0/3203229992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a7804ff20 0x7f6a78005f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6a78007a00 tx=0x7f6a6c0091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=156e50ffa5567ffd server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.394+0000 7fcedea00640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee00afdd0 0x7fcee00a7050 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fced000cdb0 tx=0x7fced0001b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.394+0000 7f6a7e000640 1 -- 172.21.15.16:0/3203229992 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c02ace0 con 0x7f6a7804ff20 2024-08-17T02:45:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.394+0000 7f6a7e000640 1 -- 172.21.15.16:0/3203229992 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c02ae40 con 0x7f6a7804ff20 2024-08-17T02:45:28.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.395+0000 7f3b4d600640 1 Processor -- start 2024-08-17T02:45:28.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.394+0000 7f6a7e000640 1 -- 172.21.15.16:0/3203229992 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c032670 con 0x7f6a7804ff20 2024-08-17T02:45:28.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.395+0000 7fced6a00640 1 -- 172.21.15.16:0/669682288 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced000a850 con 0x7fcee00afdd0 2024-08-17T02:45:28.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.395+0000 7fced6a00640 1 -- 172.21.15.16:0/669682288 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced000a9b0 con 0x7fcee00afdd0 2024-08-17T02:45:28.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.395+0000 7fced6a00640 1 -- 172.21.15.16:0/669682288 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced000a850 con 0x7fcee00afdd0 2024-08-17T02:45:28.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.396+0000 7f6a85200640 1 -- 172.21.15.16:0/3203229992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a7804ff20 msgr2=0x7f6a78005f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.396+0000 7f6a85200640 1 --2- 172.21.15.16:0/3203229992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a7804ff20 0x7f6a78005f00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6a78007a00 tx=0x7f6a6c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.396+0000 7fcedfe00640 1 -- 172.21.15.16:0/669682288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee0148740 con 0x7fcee00afdd0 2024-08-17T02:45:28.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.396+0000 7f6a85200640 1 -- 172.21.15.16:0/3203229992 shutdown_connections 2024-08-17T02:45:28.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.396+0000 7f6a85200640 1 --2- 172.21.15.16:0/3203229992 >> v2:172.21.15.16:3300/0 conn(0x7f6a78006440 0x7f6a780ea010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.396+0000 7f6a85200640 1 --2- 172.21.15.16:0/3203229992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a7804ff20 0x7f6a78005f00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.396+0000 7f6a85200640 1 --2- 172.21.15.16:0/3203229992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 0x7f6a7804f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.396+0000 7f6a85200640 1 -- 172.21.15.16:0/3203229992 >> 172.21.15.16:0/3203229992 conn(0x7f6a78009fa0 msgr2=0x7f6a7800a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.395+0000 7f3b4d600640 1 -- start start 2024-08-17T02:45:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.396+0000 7fcedfe00640 1 -- 172.21.15.16:0/669682288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee00a7590 con 0x7fcee00afdd0 2024-08-17T02:45:28.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7f2c37800640 1 Processor -- start 2024-08-17T02:45:28.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7fd901600640 1 Processor -- start 2024-08-17T02:45:28.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7f6a85200640 1 -- 172.21.15.16:0/3203229992 shutdown_connections 2024-08-17T02:45:28.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7fd901600640 1 -- start start 2024-08-17T02:45:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7f2c37800640 1 -- start start 2024-08-17T02:45:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7f3b4d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b480ada10 0x7f3b480b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7f3b4d600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3b480b1120 0x7f3b480b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7f3b4d600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3b480b36e0 0x7f3b480adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7f3b4d600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b480d0ab0 con 0x7f3b480b36e0 2024-08-17T02:45:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7f3b4d600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b480b00c0 con 0x7f3b480b1120 2024-08-17T02:45:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7f3b4d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480b0360 con 0x7f3b480ada10 2024-08-17T02:45:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7f3b46a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b480ada10 0x7f3b480b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.398+0000 7f3b46a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b480ada10 0x7f3b480b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:53198/0 (socket says 172.21.15.16:53198) 2024-08-17T02:45:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.398+0000 7f3b46a00640 1 -- 172.21.15.16:0/1702324905 learned_addr learned my addr 172.21.15.16:0/1702324905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.398+0000 7f3b46a00640 1 -- 172.21.15.16:0/1702324905 >> v2:172.21.15.142:3300/0 conn(0x7f3b480b1120 msgr2=0x7f3b480b31a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:45:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.398+0000 7f3b46a00640 1 --2- 172.21.15.16:0/1702324905 >> v2:172.21.15.142:3300/0 conn(0x7f3b480b1120 0x7f3b480b31a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.398+0000 7f3b46a00640 1 -- 172.21.15.16:0/1702324905 >> v2:172.21.15.16:3300/0 conn(0x7f3b480b36e0 msgr2=0x7f3b480adde0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-17T02:45:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.398+0000 7f2c37800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c280091c0 0x7f2c280f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.398+0000 7f2c37800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0860 0x7f2c280da8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.398+0000 7f2c37800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c280e0c30 0x7f2c28053be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.399+0000 7f2c37800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2804ea40 con 0x7f2c280e0c30 2024-08-17T02:45:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.399+0000 7f2c37800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2804e110 con 0x7f2c280091c0 2024-08-17T02:45:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.399+0000 7f2c37800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2804e3f0 con 0x7f2c280e0860 2024-08-17T02:45:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.399+0000 7f3b47400640 1 --2- 172.21.15.16:0/1702324905 >> v2:172.21.15.16:3300/0 conn(0x7f3b480b36e0 0x7f3b480adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.399+0000 7f2c34e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c280091c0 0x7f2c280f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7fd901600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8fc06a8d0 0x7fd8fc157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.399+0000 7f2c35800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c280e0c30 0x7f2c28053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.399+0000 7f2c34e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c280091c0 0x7f2c280f77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36910/0 (socket says 172.21.15.16:36910) 2024-08-17T02:45:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.399+0000 7f2c34e00640 1 -- 172.21.15.16:0/1263350785 learned_addr learned my addr 172.21.15.16:0/1263350785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.399+0000 7f2c2fe00640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0860 0x7f2c280da8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.397+0000 7f6a85200640 1 -- 172.21.15.16:0/3203229992 wait complete. 2024-08-17T02:45:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7f6a85200640 1 Processor -- start 2024-08-17T02:45:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7fd901600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8fc06bf30 0x7fd8fc134d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7fd8faa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8fc06a8d0 0x7fd8fc157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7fd901600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8fc13f4a0 0x7fd8fc16ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7fd8faa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8fc06a8d0 0x7fd8fc157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:53212/0 (socket says 172.21.15.16:53212) 2024-08-17T02:45:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7fd8faa00640 1 -- 172.21.15.16:0/4093858912 learned_addr learned my addr 172.21.15.16:0/4093858912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7fd8fa000640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8fc06bf30 0x7fd8fc134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.401+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc134a00 con 0x7fd8fc06bf30 2024-08-17T02:45:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.401+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc156350 con 0x7fd8fc13f4a0 2024-08-17T02:45:28.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.401+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc134440 con 0x7fd8fc06a8d0 2024-08-17T02:45:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.401+0000 7fd8fb400640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8fc13f4a0 0x7fd8fc16ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7f6a85200640 1 -- start start 2024-08-17T02:45:28.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.401+0000 7fcedfe00640 1 -- 172.21.15.16:0/669682288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee0071050 con 0x7fcee00afdd0 2024-08-17T02:45:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7f2c2fe00640 1 -- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c280091c0 msgr2=0x7f2c280f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7f2c2fe00640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c280091c0 0x7f2c280f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7f2c2fe00640 1 -- 172.21.15.16:0/1263350785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c280e0c30 msgr2=0x7f2c28053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7f2c2fe00640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c280e0c30 0x7f2c28053be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7f2c2fe00640 1 -- 172.21.15.16:0/1263350785 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c24009450 con 0x7f2c280e0860 2024-08-17T02:45:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7f2c34e00640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c280091c0 0x7f2c280f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7f2c2fe00640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0860 0x7f2c280da8f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c24031000 tx=0x7f2c24032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7f2c23e00640 1 -- 172.21.15.16:0/1263350785 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24031c80 con 0x7f2c280e0860 2024-08-17T02:45:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7f2c23e00640 1 -- 172.21.15.16:0/1263350785 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c24031e00 con 0x7f2c280e0860 2024-08-17T02:45:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7f2c23e00640 1 -- 172.21.15.16:0/1263350785 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2403d670 con 0x7f2c280e0860 2024-08-17T02:45:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7f2c37800640 1 -- 172.21.15.16:0/1263350785 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2804e6d0 con 0x7f2c280e0860 2024-08-17T02:45:28.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7f2c37800640 1 -- 172.21.15.16:0/1263350785 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c280f74d0 con 0x7f2c280e0860 2024-08-17T02:45:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7fd8fa000640 1 -- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8fc06a8d0 msgr2=0x7fd8fc157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7fd8fa000640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8fc06a8d0 0x7fd8fc157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7fd8fa000640 1 -- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8fc13f4a0 msgr2=0x7fd8fc16ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7fd8fa000640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8fc13f4a0 0x7fd8fc16ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7fd8fa000640 1 -- 172.21.15.16:0/4093858912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec009450 con 0x7fd8fc06bf30 2024-08-17T02:45:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7fd8fb400640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8fc13f4a0 0x7fd8fc16ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:28.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.403+0000 7fd8fa000640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8fc06bf30 0x7fd8fc134d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec031000 tx=0x7fd8ec035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.403+0000 7fd8f2000640 1 -- 172.21.15.16:0/4093858912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec030a60 con 0x7fd8fc06bf30 2024-08-17T02:45:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.403+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc134720 con 0x7fd8fc06bf30 2024-08-17T02:45:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7f3b46a00640 1 --2- 172.21.15.16:0/1702324905 >> v2:172.21.15.16:3300/0 conn(0x7f3b480b36e0 0x7f3b480adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.403+0000 7fd8f2000640 1 -- 172.21.15.16:0/4093858912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec030bc0 con 0x7fd8fc06bf30 2024-08-17T02:45:28.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.403+0000 7fd8f2000640 1 -- 172.21.15.16:0/4093858912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec030a60 con 0x7fd8fc06bf30 2024-08-17T02:45:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.400+0000 7f3b46a00640 1 -- 172.21.15.16:0/1702324905 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b480b0640 con 0x7f3b480ada10 2024-08-17T02:45:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.404+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc158450 con 0x7fd8fc06bf30 2024-08-17T02:45:28.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.404+0000 7f6a85200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78006440 0x7f6a780e0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.401+0000 7f3b46a00640 1 --2- 172.21.15.16:0/1702324905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b480ada10 0x7f3b480b2c60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3b48064270 tx=0x7f3b40009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a1d1ad47157a1d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.404+0000 7f6a85200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a7804ff20 0x7f6a780c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.404+0000 7f6a85200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 0x7f6a780c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7f3b45600640 1 -- 172.21.15.16:0/1702324905 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40031400 con 0x7f3b480ada10 2024-08-17T02:45:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.404+0000 7f6a85200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780deb80 con 0x7f6a7804ff20 2024-08-17T02:45:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.404+0000 7f6a85200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780ddf70 con 0x7f6a78006440 2024-08-17T02:45:28.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.404+0000 7f6a85200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780de250 con 0x7f6a780e09f0 2024-08-17T02:45:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.404+0000 7f6a7fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 0x7f6a780c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.404+0000 7f6a7fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 0x7f6a780c6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:53228/0 (socket says 172.21.15.16:53228) 2024-08-17T02:45:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.404+0000 7f6a7fe00640 1 -- 172.21.15.16:0/3086023113 learned_addr learned my addr 172.21.15.16:0/3086023113 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7f3b45600640 1 -- 172.21.15.16:0/1702324905 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4002d050 con 0x7f3b480ada10 2024-08-17T02:45:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.402+0000 7f3b45600640 1 -- 172.21.15.16:0/1702324905 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40031790 con 0x7f3b480ada10 2024-08-17T02:45:28.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.403+0000 7f3b4d600640 1 -- 172.21.15.16:0/1702324905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b480ada10 msgr2=0x7f3b480b2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.403+0000 7f3b4d600640 1 --2- 172.21.15.16:0/1702324905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b480ada10 0x7f3b480b2c60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3b48064270 tx=0x7f3b40009840 comp rx=0 tx=0).stop 2024-08-17T02:45:28.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.405+0000 7f6a7f400640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78006440 0x7f6a780e0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.403+0000 7f3b47400640 1 --2- 172.21.15.16:0/1702324905 >> v2:172.21.15.16:3300/0 conn(0x7f3b480b36e0 0x7f3b480adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:28.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.405+0000 7f3b4d600640 1 -- 172.21.15.16:0/1702324905 shutdown_connections 2024-08-17T02:45:28.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.405+0000 7f3b4d600640 1 --2- 172.21.15.16:0/1702324905 >> v2:172.21.15.16:3300/0 conn(0x7f3b480b36e0 0x7f3b480adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.405+0000 7f3b4d600640 1 --2- 172.21.15.16:0/1702324905 >> v2:172.21.15.142:3300/0 conn(0x7f3b480b1120 0x7f3b480b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.405+0000 7f3b4d600640 1 --2- 172.21.15.16:0/1702324905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b480ada10 0x7f3b480b2c60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.405+0000 7f3b4d600640 1 -- 172.21.15.16:0/1702324905 >> 172.21.15.16:0/1702324905 conn(0x7f3b480cfdf0 msgr2=0x7f3b480d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.405+0000 7f6a7fe00640 1 -- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78006440 msgr2=0x7f6a780e0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.405+0000 7f3b4d600640 1 -- 172.21.15.16:0/1702324905 shutdown_connections 2024-08-17T02:45:28.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.406+0000 7f6a7ea00640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a7804ff20 0x7f6a780c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.406+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8fc156120 con 0x7fd8fc06bf30 2024-08-17T02:45:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.407+0000 7f6a7fe00640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78006440 0x7f6a780e0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.407+0000 7f6a7fe00640 1 -- 172.21.15.16:0/3086023113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a7804ff20 msgr2=0x7f6a780c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.407+0000 7f6a7fe00640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a7804ff20 0x7f6a780c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.407+0000 7f6a7fe00640 1 -- 172.21.15.16:0/3086023113 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c009450 con 0x7f6a780e09f0 2024-08-17T02:45:28.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.407+0000 7f6a7fe00640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 0x7f6a780c6970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6a800661b0 tx=0x7f6a80069040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.406+0000 7f2c23e00640 1 -- 172.21.15.16:0/1263350785 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f2c24039040 con 0x7f2c280e0860 2024-08-17T02:45:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.406+0000 7f2c37800640 1 -- 172.21.15.16:0/1263350785 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0c005210 con 0x7f2c280e0860 2024-08-17T02:45:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.407+0000 7f2c23e00640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2c10065220 0x7f2c100676d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.407+0000 7f2c23e00640 1 -- 172.21.15.16:0/1263350785 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2c240a2560 con 0x7f2c280e0860 2024-08-17T02:45:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.407+0000 7f2c34e00640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2c10065220 0x7f2c100676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.405+0000 7f3b4d600640 1 -- 172.21.15.16:0/1702324905 wait complete. 2024-08-17T02:45:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.408+0000 7f6a7f400640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78006440 0x7f6a780e0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.408+0000 7f6a7ea00640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a7804ff20 0x7f6a780c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.408+0000 7f3b4d600640 1 Processor -- start 2024-08-17T02:45:28.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.408+0000 7f2c34e00640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2c10065220 0x7f2c100676d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c006fd0 tx=0x7f2c1c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.406+0000 7fced6a00640 1 -- 172.21.15.16:0/669682288 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fced00070f0 con 0x7fcee00afdd0 2024-08-17T02:45:28.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.408+0000 7fced6a00640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcebc065230 0x7fcebc0676e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.408+0000 7fced6a00640 1 -- 172.21.15.16:0/669682288 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fced0082330 con 0x7fcee00afdd0 2024-08-17T02:45:28.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.408+0000 7f3b4d600640 1 -- start start 2024-08-17T02:45:28.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.409+0000 7fd8f2000640 1 -- 172.21.15.16:0/4093858912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fd8ec045460 con 0x7fd8fc06bf30 2024-08-17T02:45:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.409+0000 7fced6000640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcebc065230 0x7fcebc0676e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.410+0000 7fced6000640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcebc065230 0x7fcebc0676e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcec8000f80 tx=0x7fcec8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.410+0000 7fd8f2000640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd8dc0651b0 0x7fd8dc067660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.410+0000 7fd8f2000640 1 -- 172.21.15.16:0/4093858912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd8ec0a35a0 con 0x7fd8fc06bf30 2024-08-17T02:45:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f3b4d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b480ada10 0x7f3b4813dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f3b4d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b480b1120 0x7f3b480ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7fd8faa00640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd8dc0651b0 0x7fd8dc067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f6a72a00640 1 -- 172.21.15.16:0/3086023113 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800711a0 con 0x7f6a780e09f0 2024-08-17T02:45:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f3b46a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b480ada10 0x7f3b4813dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f3b4d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b480b36e0 0x7f3b4814c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f6a72a00640 1 -- 172.21.15.16:0/3086023113 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a80064110 con 0x7f6a780e09f0 2024-08-17T02:45:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f6a72a00640 1 -- 172.21.15.16:0/3086023113 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8006d6a0 con 0x7f6a780e09f0 2024-08-17T02:45:28.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f3b4d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4809a540 con 0x7f3b480b1120 2024-08-17T02:45:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f3b4d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480cefa0 con 0x7f3b480ada10 2024-08-17T02:45:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f6a85200640 1 -- 172.21.15.16:0/3086023113 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a780de530 con 0x7f6a780e09f0 2024-08-17T02:45:28.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f6a85200640 1 -- 172.21.15.16:0/3086023113 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a780ddae0 con 0x7f6a780e09f0 2024-08-17T02:45:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f3b4d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b48099c10 con 0x7f3b480b36e0 2024-08-17T02:45:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f3b46a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b480ada10 0x7f3b4813dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:36936/0 (socket says 172.21.15.16:36936) 2024-08-17T02:45:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f3b46000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b480b1120 0x7f3b480ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.411+0000 7f3b46000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b480b1120 0x7f3b480ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58278/0 (socket says 172.21.15.16:58278) 2024-08-17T02:45:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.412+0000 7f3b46000640 1 -- 172.21.15.16:0/3746671383 learned_addr learned my addr 172.21.15.16:0/3746671383 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.412+0000 7f3b46000640 1 -- 172.21.15.16:0/3746671383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b480b36e0 msgr2=0x7f3b4814c550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-17T02:45:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.412+0000 7f3b46000640 1 --2- 172.21.15.16:0/3746671383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b480b36e0 0x7f3b4814c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.413+0000 7fced6a00640 1 -- 172.21.15.16:0/669682288 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fced00503d0 con 0x7fcee00afdd0 2024-08-17T02:45:28.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.413+0000 7f2c23e00640 1 -- 172.21.15.16:0/1263350785 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2c24070600 con 0x7f2c280e0860 2024-08-17T02:45:28.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.412+0000 7f3b46000640 1 -- 172.21.15.16:0/3746671383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b480ada10 msgr2=0x7f3b4813dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.412+0000 7f3b46000640 1 --2- 172.21.15.16:0/3746671383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b480ada10 0x7f3b4813dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.413+0000 7f3b46000640 1 -- 172.21.15.16:0/3746671383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b40009ae0 con 0x7f3b480b1120 2024-08-17T02:45:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.413+0000 7f3b46000640 1 --2- 172.21.15.16:0/3746671383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b480b1120 0x7f3b480ce550 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400b5e0 tx=0x7f3b340090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.414+0000 7f6a85200640 1 -- 172.21.15.16:0/3086023113 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a780ddd30 con 0x7f6a780e09f0 2024-08-17T02:45:28.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.416+0000 7fd8faa00640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd8dc0651b0 0x7fd8dc067660 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009ee0 tx=0x7fd8f4009840 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.416+0000 7f6a72a00640 1 -- 172.21.15.16:0/3086023113 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f6a8006dab0 con 0x7f6a780e09f0 2024-08-17T02:45:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.417+0000 7f6a72a00640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6a600652a0 0x7f6a60067750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.417+0000 7f6a72a00640 1 -- 172.21.15.16:0/3086023113 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6a8008b070 con 0x7f6a780e09f0 2024-08-17T02:45:28.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.418+0000 7f6a7f400640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6a600652a0 0x7f6a60067750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.418+0000 7f3b3e000640 1 -- 172.21.15.16:0/3746671383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400f590 con 0x7f3b480b1120 2024-08-17T02:45:28.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.418+0000 7f3b3e000640 1 -- 172.21.15.16:0/3746671383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3400f6f0 con 0x7f3b480b1120 2024-08-17T02:45:28.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.418+0000 7f3b3e000640 1 -- 172.21.15.16:0/3746671383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400faf0 con 0x7f3b480b1120 2024-08-17T02:45:28.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.419+0000 7f3b4d600640 1 -- 172.21.15.16:0/3746671383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b48099ef0 con 0x7f3b480b1120 2024-08-17T02:45:28.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.419+0000 7f3b4d600640 1 -- 172.21.15.16:0/3746671383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4809d010 con 0x7f3b480b1120 2024-08-17T02:45:28.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.419+0000 7fd8f2000640 1 -- 172.21.15.16:0/4093858912 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd8fc156120 con 0x7fd8fc06bf30 2024-08-17T02:45:28.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.420+0000 7f6a7f400640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6a600652a0 0x7f6a60067750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6a74008a80 tx=0x7f6a74008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.421+0000 7f3b4d600640 1 -- 172.21.15.16:0/3746671383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b20005210 con 0x7f3b480b1120 2024-08-17T02:45:28.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.421+0000 7f6a72a00640 1 -- 172.21.15.16:0/3086023113 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6a780ddd30 con 0x7f6a780e09f0 2024-08-17T02:45:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.423+0000 7f3b3e000640 1 -- 172.21.15.16:0/3746671383 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f3b3401a460 con 0x7f3b480b1120 2024-08-17T02:45:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.424+0000 7f3b3e000640 1 --2- 172.21.15.16:0/3746671383 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3b280652a0 0x7f3b28067750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.424+0000 7f3b3e000640 1 -- 172.21.15.16:0/3746671383 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3b34081fe0 con 0x7f3b480b1120 2024-08-17T02:45:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.427+0000 7f3b46a00640 1 --2- 172.21.15.16:0/3746671383 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3b280652a0 0x7f3b28067750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:28.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.431+0000 7f3b46a00640 1 --2- 172.21.15.16:0/3746671383 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3b280652a0 0x7f3b28067750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b40033000 tx=0x7f3b40036000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:28.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.433+0000 7f3b3e000640 1 -- 172.21.15.16:0/3746671383 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f3b34050100 con 0x7f3b480b1120 2024-08-17T02:45:28.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.740+0000 7f1bc7e00640 1 -- 172.21.15.16:0/167247389 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1ba0005790 con 0x7f1bac005250 2024-08-17T02:45:28.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.741+0000 7f1bbe000640 1 -- 172.21.15.16:0/167247389 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1bb0022020 con 0x7f1bac005250 2024-08-17T02:45:28.742 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.743+0000 7f4898c00640 1 -- 172.21.15.16:0/1082119407 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4894071050 con 0x7f4894156130 2024-08-17T02:45:28.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.744+0000 7f488b400640 1 -- 172.21.15.16:0/1082119407 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f488c053b00 con 0x7f4894156130 2024-08-17T02:45:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.745+0000 7f17f6a00640 1 -- 172.21.15.16:0/652472802 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f17cc0054c0 con 0x7f17e80e01d0 2024-08-17T02:45:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.744+0000 7f1bc7e00640 1 -- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1ba4065170 msgr2=0x7f1ba4067620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.744+0000 7f1bc7e00640 1 --2- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1ba4065170 0x7f1ba4067620 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8015fd0 tx=0x7f1bb8017040 comp rx=0 tx=0).stop 2024-08-17T02:45:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.745+0000 7f1bc7e00640 1 -- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 msgr2=0x7f1bc8149930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.745+0000 7f1bc7e00640 1 --2- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 0x7f1bc8149930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0011a60 tx=0x7f1bb00090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.747 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2024-08-17T02:45:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.745+0000 7f1bc7e00640 1 -- 172.21.15.16:0/167247389 shutdown_connections 2024-08-17T02:45:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.745+0000 7f1bc7e00640 1 --2- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1ba4065170 0x7f1ba4067620 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.745+0000 7f1bc7e00640 1 --2- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1bc806d8b0 0x7f1bc813ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.745+0000 7f1bc7e00640 1 --2- 172.21.15.16:0/167247389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bc814e1d0 0x7f1bc81690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.745+0000 7f1bc7e00640 1 --2- 172.21.15.16:0/167247389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1bac005250 0x7f1bc8149930 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.745+0000 7f1bc7e00640 1 -- 172.21.15.16:0/167247389 >> 172.21.15.16:0/167247389 conn(0x7f1bc80cb790 msgr2=0x7f1bc80ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.745+0000 7f17e3400640 1 -- 172.21.15.16:0/652472802 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f17dc053540 con 0x7f17e80e01d0 2024-08-17T02:45:28.750 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2024-08-17T02:45:28.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.745+0000 7f1bc7e00640 1 -- 172.21.15.16:0/167247389 shutdown_connections 2024-08-17T02:45:28.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.746+0000 7f1bc7e00640 1 -- 172.21.15.16:0/167247389 wait complete. 2024-08-17T02:45:28.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.747+0000 7f4898c00640 1 -- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4870065230 msgr2=0x7f48700676e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.747+0000 7f4898c00640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4870065230 0x7f48700676e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4884038000 tx=0x7f488403b000 comp rx=0 tx=0).stop 2024-08-17T02:45:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.747+0000 7f4898c00640 1 -- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4894156130 msgr2=0x7f4894154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.747+0000 7f4898c00640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4894156130 0x7f4894154240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f488c00d890 tx=0x7f488c008040 comp rx=0 tx=0).stop 2024-08-17T02:45:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.748+0000 7f4898c00640 1 -- 172.21.15.16:0/1082119407 shutdown_connections 2024-08-17T02:45:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.748+0000 7f4898c00640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4870065230 0x7f48700676e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.748+0000 7f4898c00640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4894156130 0x7f4894154240 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.748+0000 7f4898c00640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4894152960 0x7f4894153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.748+0000 7f4898c00640 1 --2- 172.21.15.16:0/1082119407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48940af7c0 0x7f4894150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.748+0000 7f4898c00640 1 -- 172.21.15.16:0/1082119407 >> 172.21.15.16:0/1082119407 conn(0x7f48940cb790 msgr2=0x7f48940cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.748+0000 7f4898c00640 1 -- 172.21.15.16:0/1082119407 shutdown_connections 2024-08-17T02:45:28.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.748+0000 7f4898c00640 1 -- 172.21.15.16:0/1082119407 wait complete. 2024-08-17T02:45:28.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.749+0000 7f17f6a00640 1 -- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f17d00652a0 msgr2=0x7f17d0067750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.749+0000 7f17f6a00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f17d00652a0 0x7f17d0067750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f17e4034000 tx=0x7f17e4032000 comp rx=0 tx=0).stop 2024-08-17T02:45:28.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.749+0000 7f17f6a00640 1 -- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17e80e01d0 msgr2=0x7f17e80ee3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.749+0000 7f17f6a00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17e80e01d0 0x7f17e80ee3e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00c5b0 tx=0x7f17dc007d00 comp rx=0 tx=0).stop 2024-08-17T02:45:28.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.750+0000 7f17f6a00640 1 -- 172.21.15.16:0/652472802 shutdown_connections 2024-08-17T02:45:28.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.750+0000 7f17f6a00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f17d00652a0 0x7f17d0067750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.750+0000 7f17f6a00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f17e80e01d0 0x7f17e80ee3e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.750+0000 7f17f6a00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17e80358c0 0x7f17e804fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.750+0000 7f17f6a00640 1 --2- 172.21.15.16:0/652472802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f17e800da90 0x7f17e80e30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.750+0000 7f17f6a00640 1 -- 172.21.15.16:0/652472802 >> 172.21.15.16:0/652472802 conn(0x7f17e8057870 msgr2=0x7f17e8055f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.751+0000 7f17f6a00640 1 -- 172.21.15.16:0/652472802 shutdown_connections 2024-08-17T02:45:28.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.751+0000 7f17f6a00640 1 -- 172.21.15.16:0/652472802 wait complete. 2024-08-17T02:45:28.774 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2024-08-17T02:45:28.775 DEBUG:teuthology.parallel:result is None 2024-08-17T02:45:28.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-08-17T02:45:28.778 DEBUG:teuthology.parallel:result is None 2024-08-17T02:45:28.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-08-17T02:45:28.778 DEBUG:teuthology.parallel:result is None 2024-08-17T02:45:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.801+0000 7fcedfe00640 1 -- 172.21.15.16:0/669682288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fcee01482c0 con 0x7fcee00afdd0 2024-08-17T02:45:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.802+0000 7fced6a00640 1 -- 172.21.15.16:0/669682288 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcee01482c0 con 0x7fcee00afdd0 2024-08-17T02:45:28.803 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.802+0000 7f6a85200640 1 -- 172.21.15.16:0/3086023113 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6a780ddd30 con 0x7f6a780e09f0 2024-08-17T02:45:28.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.802+0000 7f6a72a00640 1 -- 172.21.15.16:0/3086023113 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6a80095020 con 0x7f6a780e09f0 2024-08-17T02:45:28.804 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.804+0000 7fcedfe00640 1 -- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcebc065230 msgr2=0x7fcebc0676e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.804+0000 7fcedfe00640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcebc065230 0x7fcebc0676e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcec8000f80 tx=0x7fcec8009040 comp rx=0 tx=0).stop 2024-08-17T02:45:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.804+0000 7fcedfe00640 1 -- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee00afdd0 msgr2=0x7fcee00a7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.804+0000 7fcedfe00640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee00afdd0 0x7fcee00a7050 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fced000cdb0 tx=0x7fced0001b70 comp rx=0 tx=0).stop 2024-08-17T02:45:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.804+0000 7fcedfe00640 1 -- 172.21.15.16:0/669682288 shutdown_connections 2024-08-17T02:45:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.804+0000 7fcedfe00640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcebc065230 0x7fcebc0676e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.804+0000 7fcedfe00640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcee0155900 0x7fcee0148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.804+0000 7fcedfe00640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcee00b0270 0x7fcee0167f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.804+0000 7fcedfe00640 1 --2- 172.21.15.16:0/669682288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcee00afdd0 0x7fcee00a7050 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.804+0000 7fcedfe00640 1 -- 172.21.15.16:0/669682288 >> 172.21.15.16:0/669682288 conn(0x7fcee0067750 msgr2=0x7fcee006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.804+0000 7fcedfe00640 1 -- 172.21.15.16:0/669682288 shutdown_connections 2024-08-17T02:45:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.804+0000 7fcedfe00640 1 -- 172.21.15.16:0/669682288 wait complete. 2024-08-17T02:45:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.806+0000 7f6a85200640 1 -- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6a600652a0 msgr2=0x7f6a60067750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.806+0000 7f6a85200640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6a600652a0 0x7f6a60067750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6a74008a80 tx=0x7f6a74008040 comp rx=0 tx=0).stop 2024-08-17T02:45:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.806+0000 7f6a85200640 1 -- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 msgr2=0x7f6a780c6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.806+0000 7f6a85200640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 0x7f6a780c6970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6a800661b0 tx=0x7f6a80069040 comp rx=0 tx=0).stop 2024-08-17T02:45:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.806+0000 7f6a85200640 1 -- 172.21.15.16:0/3086023113 shutdown_connections 2024-08-17T02:45:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.806+0000 7f6a85200640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6a600652a0 0x7f6a60067750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.806+0000 7f6a85200640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6a780e09f0 0x7f6a780c6970 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.806+0000 7f6a85200640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a7804ff20 0x7f6a780c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.806+0000 7f6a85200640 1 --2- 172.21.15.16:0/3086023113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a78006440 0x7f6a780e0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.806+0000 7f6a85200640 1 -- 172.21.15.16:0/3086023113 >> 172.21.15.16:0/3086023113 conn(0x7f6a78009fa0 msgr2=0x7f6a780e96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.806+0000 7f6a85200640 1 -- 172.21.15.16:0/3086023113 shutdown_connections 2024-08-17T02:45:28.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.806+0000 7f6a85200640 1 -- 172.21.15.16:0/3086023113 wait complete. 2024-08-17T02:45:28.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.811+0000 7f3b4d600640 1 -- 172.21.15.16:0/3746671383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3b200054c0 con 0x7f3b480b1120 2024-08-17T02:45:28.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.812+0000 7f3b3e000640 1 -- 172.21.15.16:0/3746671383 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3b34053720 con 0x7f3b480b1120 2024-08-17T02:45:28.813 INFO:teuthology.orchestra.run.smithi016.stdout:55834574853 2024-08-17T02:45:28.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.815+0000 7f3b4d600640 1 -- 172.21.15.16:0/3746671383 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3b280652a0 msgr2=0x7f3b28067750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.815+0000 7f3b4d600640 1 --2- 172.21.15.16:0/3746671383 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3b280652a0 0x7f3b28067750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b40033000 tx=0x7f3b40036000 comp rx=0 tx=0).stop 2024-08-17T02:45:28.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.815+0000 7f3b4d600640 1 -- 172.21.15.16:0/3746671383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b480b1120 msgr2=0x7f3b480ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.815+0000 7f3b4d600640 1 --2- 172.21.15.16:0/3746671383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b480b1120 0x7f3b480ce550 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400b5e0 tx=0x7f3b340090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.815+0000 7f3b4d600640 1 -- 172.21.15.16:0/3746671383 shutdown_connections 2024-08-17T02:45:28.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.815+0000 7f3b4d600640 1 --2- 172.21.15.16:0/3746671383 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3b280652a0 0x7f3b28067750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.815+0000 7f3b4d600640 1 --2- 172.21.15.16:0/3746671383 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b480b36e0 0x7f3b4814c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.815+0000 7f3b4d600640 1 --2- 172.21.15.16:0/3746671383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b480b1120 0x7f3b480ce550 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.815+0000 7f3b4d600640 1 --2- 172.21.15.16:0/3746671383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b480ada10 0x7f3b4813dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.815+0000 7f3b4d600640 1 -- 172.21.15.16:0/3746671383 >> 172.21.15.16:0/3746671383 conn(0x7f3b480cfdf0 msgr2=0x7f3b480b3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.816+0000 7f3b4d600640 1 -- 172.21.15.16:0/3746671383 shutdown_connections 2024-08-17T02:45:28.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.816+0000 7f3b4d600640 1 -- 172.21.15.16:0/3746671383 wait complete. 2024-08-17T02:45:28.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2024-08-17T02:45:28.822 DEBUG:teuthology.parallel:result is None 2024-08-17T02:45:28.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.7 2024-08-17T02:45:28.830 DEBUG:teuthology.parallel:result is None 2024-08-17T02:45:28.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-08-17T02:45:28.835 DEBUG:teuthology.parallel:result is None 2024-08-17T02:45:28.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.838+0000 7f2c37800640 1 -- 172.21.15.16:0/1263350785 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2c0c0054c0 con 0x7f2c280e0860 2024-08-17T02:45:28.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.839+0000 7f2c23e00640 1 -- 172.21.15.16:0/1263350785 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2c24073c20 con 0x7f2c280e0860 2024-08-17T02:45:28.840 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:28.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.841+0000 7f2c37800640 1 -- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2c10065220 msgr2=0x7f2c100676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.841+0000 7f2c37800640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2c10065220 0x7f2c100676d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c006fd0 tx=0x7f2c1c008040 comp rx=0 tx=0).stop 2024-08-17T02:45:28.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.841+0000 7f2c37800640 1 -- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0860 msgr2=0x7f2c280da8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.841+0000 7f2c37800640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0860 0x7f2c280da8f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2c24031000 tx=0x7f2c24032000 comp rx=0 tx=0).stop 2024-08-17T02:45:28.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.841+0000 7f2c37800640 1 -- 172.21.15.16:0/1263350785 shutdown_connections 2024-08-17T02:45:28.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.841+0000 7f2c37800640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2c10065220 0x7f2c100676d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.841+0000 7f2c37800640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c280e0c30 0x7f2c28053be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.841+0000 7f2c37800640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2c280e0860 0x7f2c280da8f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.841+0000 7f2c37800640 1 --2- 172.21.15.16:0/1263350785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2c280091c0 0x7f2c280f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.841+0000 7f2c37800640 1 -- 172.21.15.16:0/1263350785 >> 172.21.15.16:0/1263350785 conn(0x7f2c28052710 msgr2=0x7f2c280e3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.841+0000 7f2c37800640 1 -- 172.21.15.16:0/1263350785 shutdown_connections 2024-08-17T02:45:28.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.841+0000 7f2c37800640 1 -- 172.21.15.16:0/1263350785 wait complete. 2024-08-17T02:45:28.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.842+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd8fc156120 con 0x7fd8fc06bf30 2024-08-17T02:45:28.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.843+0000 7fd8f2000640 1 -- 172.21.15.16:0/4093858912 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd8fc156120 con 0x7fd8fc06bf30 2024-08-17T02:45:28.845 INFO:teuthology.orchestra.run.smithi016.stdout:55834574852 2024-08-17T02:45:28.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.845+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd8dc0651b0 msgr2=0x7fd8dc067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.845+0000 7fd901600640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd8dc0651b0 0x7fd8dc067660 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009ee0 tx=0x7fd8f4009840 comp rx=0 tx=0).stop 2024-08-17T02:45:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.845+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8fc06bf30 msgr2=0x7fd8fc134d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.845+0000 7fd901600640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8fc06bf30 0x7fd8fc134d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec031000 tx=0x7fd8ec035000 comp rx=0 tx=0).stop 2024-08-17T02:45:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.845+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 shutdown_connections 2024-08-17T02:45:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.845+0000 7fd901600640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd8dc0651b0 0x7fd8dc067660 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.845+0000 7fd901600640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8fc13f4a0 0x7fd8fc16ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.845+0000 7fd901600640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8fc06bf30 0x7fd8fc134d90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.845+0000 7fd901600640 1 --2- 172.21.15.16:0/4093858912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8fc06a8d0 0x7fd8fc157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.845+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 >> 172.21.15.16:0/4093858912 conn(0x7fd8fc06d8f0 msgr2=0x7fd8fc0ae030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.845+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 shutdown_connections 2024-08-17T02:45:28.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:28.845+0000 7fd901600640 1 -- 172.21.15.16:0/4093858912 wait complete. 2024-08-17T02:45:28.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.6 2024-08-17T02:45:28.861 DEBUG:teuthology.parallel:result is None 2024-08-17T02:45:28.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2024-08-17T02:45:28.865 DEBUG:teuthology.parallel:result is None 2024-08-17T02:45:28.865 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-17T02:45:28.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:45:29.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.055+0000 7f7b9ea00640 1 Processor -- start 2024-08-17T02:45:29.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.055+0000 7f7b9ea00640 1 -- start start 2024-08-17T02:45:29.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.056+0000 7f7b9ea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7b980aed50 0x7f7b980ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.056+0000 7f7b9ea00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b980ce6d0 0x7f7b980ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.056+0000 7f7b9ea00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b980cc890 con 0x7f7b980cacf0 2024-08-17T02:45:29.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.056+0000 7f7b9ea00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b980ceb20 con 0x7f7b980ce6d0 2024-08-17T02:45:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.056+0000 7f7b9ea00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b980cb4c0 con 0x7f7b980aed50 2024-08-17T02:45:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.056+0000 7f7b9cc00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b980ce6d0 0x7f7b980ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.056+0000 7f7b9cc00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b980ce6d0 0x7f7b980ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58280/0 (socket says 172.21.15.16:58280) 2024-08-17T02:45:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.056+0000 7f7b9d600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7b980aed50 0x7f7b980ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.056+0000 7f7b9cc00640 1 -- 172.21.15.16:0/1722925392 learned_addr learned my addr 172.21.15.16:0/1722925392 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:29.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.057+0000 7f7b9cc00640 1 -- 172.21.15.16:0/1722925392 >> v2:172.21.15.142:3300/0 conn(0x7f7b980aed50 msgr2=0x7f7b980ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:29.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.057+0000 7f7b9cc00640 1 --2- 172.21.15.16:0/1722925392 >> v2:172.21.15.142:3300/0 conn(0x7f7b980aed50 0x7f7b980ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.057+0000 7f7b9cc00640 1 -- 172.21.15.16:0/1722925392 >> v1:172.21.15.142:6789/0 conn(0x7f7b980cacf0 legacy=0x7f7b980c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.057+0000 7f7b9cc00640 1 -- 172.21.15.16:0/1722925392 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980cb7a0 con 0x7f7b980ce6d0 2024-08-17T02:45:29.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.058+0000 7f7b9cc00640 1 --2- 172.21.15.16:0/1722925392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b980ce6d0 0x7f7b980ca7b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b98088b90 tx=0x7f7b840091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af9a35a2ac0f565 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:29.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.058+0000 7f7b97e00640 1 -- 172.21.15.16:0/1722925392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8402ae80 con 0x7f7b980ce6d0 2024-08-17T02:45:29.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.058+0000 7f7b97e00640 1 -- 172.21.15.16:0/1722925392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b84032440 con 0x7f7b980ce6d0 2024-08-17T02:45:29.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.058+0000 7f7b97e00640 1 -- 172.21.15.16:0/1722925392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84032800 con 0x7f7b980ce6d0 2024-08-17T02:45:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.059+0000 7f7b9ea00640 1 -- 172.21.15.16:0/1722925392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b980ce6d0 msgr2=0x7f7b980ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.059+0000 7f7b9ea00640 1 --2- 172.21.15.16:0/1722925392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b980ce6d0 0x7f7b980ca7b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b98088b90 tx=0x7f7b840091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.059+0000 7f7b9ea00640 1 -- 172.21.15.16:0/1722925392 shutdown_connections 2024-08-17T02:45:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.059+0000 7f7b9ea00640 1 --2- 172.21.15.16:0/1722925392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b980ce6d0 0x7f7b980ca7b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.059+0000 7f7b9ea00640 1 --2- 172.21.15.16:0/1722925392 >> v2:172.21.15.142:3300/0 conn(0x7f7b980aed50 0x7f7b980ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.059+0000 7f7b9ea00640 1 -- 172.21.15.16:0/1722925392 >> 172.21.15.16:0/1722925392 conn(0x7f7b98098e30 msgr2=0x7f7b9806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.059+0000 7f7b9ea00640 1 -- 172.21.15.16:0/1722925392 shutdown_connections 2024-08-17T02:45:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.059+0000 7f7b9ea00640 1 -- 172.21.15.16:0/1722925392 wait complete. 2024-08-17T02:45:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.060+0000 7f7b9ea00640 1 Processor -- start 2024-08-17T02:45:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.060+0000 7f7b9ea00640 1 -- start start 2024-08-17T02:45:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.061+0000 7f7b9ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b980aed50 0x7f7b98157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.061+0000 7f7b9ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b980cacf0 0x7f7b980af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.061+0000 7f7b9ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b980ce6d0 0x7f7b980af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.061+0000 7f7b9ea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980b0a10 con 0x7f7b980cacf0 2024-08-17T02:45:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.061+0000 7f7b9ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980b0cf0 con 0x7f7b980aed50 2024-08-17T02:45:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.061+0000 7f7b9ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980b5810 con 0x7f7b980ce6d0 2024-08-17T02:45:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.061+0000 7f7b9cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b980cacf0 0x7f7b980af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.061+0000 7f7b9cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b980cacf0 0x7f7b980af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58296/0 (socket says 172.21.15.16:58296) 2024-08-17T02:45:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.061+0000 7f7b9cc00640 1 -- 172.21.15.16:0/3068653809 learned_addr learned my addr 172.21.15.16:0/3068653809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.061+0000 7f7b9d600640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b980aed50 0x7f7b98157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.061+0000 7f7b9e000640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b980ce6d0 0x7f7b980af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.062+0000 7f7b9cc00640 1 -- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b980ce6d0 msgr2=0x7f7b980af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.062+0000 7f7b9cc00640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b980ce6d0 0x7f7b980af780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.062+0000 7f7b9cc00640 1 -- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b980aed50 msgr2=0x7f7b98157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.062+0000 7f7b9cc00640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b980aed50 0x7f7b98157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.062+0000 7f7b9cc00640 1 -- 172.21.15.16:0/3068653809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b84009450 con 0x7f7b980cacf0 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.062+0000 7f7b9e000640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b980ce6d0 0x7f7b980af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.062+0000 7f7b9cc00640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b980cacf0 0x7f7b980af240 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009940 tx=0x7f7b84032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.062+0000 7f7b94c00640 1 -- 172.21.15.16:0/3068653809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84031c80 con 0x7f7b980cacf0 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.062+0000 7f7b94c00640 1 -- 172.21.15.16:0/3068653809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b84031e00 con 0x7f7b980cacf0 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.062+0000 7f7b94c00640 1 -- 172.21.15.16:0/3068653809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8403d670 con 0x7f7b980cacf0 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.062+0000 7f7b9ea00640 1 -- 172.21.15.16:0/3068653809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b980afe20 con 0x7f7b980cacf0 2024-08-17T02:45:29.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.062+0000 7f7b9ea00640 1 -- 172.21.15.16:0/3068653809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b980b03e0 con 0x7f7b980cacf0 2024-08-17T02:45:29.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.064+0000 7f7b9ea00640 1 -- 172.21.15.16:0/3068653809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b98071050 con 0x7f7b980cacf0 2024-08-17T02:45:29.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.064+0000 7f7b94c00640 1 -- 172.21.15.16:0/3068653809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f7b84039040 con 0x7f7b980cacf0 2024-08-17T02:45:29.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.066+0000 7f7b94c00640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7b78065150 0x7f7b78067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.066+0000 7f7b94c00640 1 -- 172.21.15.16:0/3068653809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7b840a26b0 con 0x7f7b980cacf0 2024-08-17T02:45:29.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.066+0000 7f7b9d600640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7b78065150 0x7f7b78067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.067+0000 7f7b9d600640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7b78065150 0x7f7b78067600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009c80 tx=0x7f7b8c0090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:29.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.070+0000 7f7b94c00640 1 -- 172.21.15.16:0/3068653809 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7b84070840 con 0x7f7b980cacf0 2024-08-17T02:45:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.248+0000 7f7b9ea00640 1 -- 172.21.15.16:0/3068653809 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b980ce370 con 0x7f7b78065150 2024-08-17T02:45:29.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.256+0000 7f7b94c00640 1 -- 172.21.15.16:0/3068653809 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264562 (secure 0 0 0) 0x7f7b980ce370 con 0x7f7b78065150 2024-08-17T02:45:29.260 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:29.261 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-17T02:45:29.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.260+0000 7f7b9ea00640 1 -- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7b78065150 msgr2=0x7f7b78067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.260+0000 7f7b9ea00640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7b78065150 0x7f7b78067600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009c80 tx=0x7f7b8c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.260+0000 7f7b9ea00640 1 -- 172.21.15.16:0/3068653809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b980cacf0 msgr2=0x7f7b980af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.260+0000 7f7b9ea00640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b980cacf0 0x7f7b980af240 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009940 tx=0x7f7b84032000 comp rx=0 tx=0).stop 2024-08-17T02:45:29.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.260+0000 7f7b9ea00640 1 -- 172.21.15.16:0/3068653809 shutdown_connections 2024-08-17T02:45:29.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.260+0000 7f7b9ea00640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7b78065150 0x7f7b78067600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.260+0000 7f7b9ea00640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7b980ce6d0 0x7f7b980af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.260+0000 7f7b9ea00640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b980cacf0 0x7f7b980af240 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.260+0000 7f7b9ea00640 1 --2- 172.21.15.16:0/3068653809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7b980aed50 0x7f7b98157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.260+0000 7f7b9ea00640 1 -- 172.21.15.16:0/3068653809 >> 172.21.15.16:0/3068653809 conn(0x7f7b98098e30 msgr2=0x7f7b98068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:29.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.260+0000 7f7b9ea00640 1 -- 172.21.15.16:0/3068653809 shutdown_connections 2024-08-17T02:45:29.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.260+0000 7f7b9ea00640 1 -- 172.21.15.16:0/3068653809 wait complete. 2024-08-17T02:45:29.276 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-08-17T02:45:28.237777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398428,"kb_used_data":2920,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752576292,"statfs":{"total":773094113280,"available":770638123008,"internally_reserved":0,"allocated":2990080,"data_stored":1787916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.224908"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048440+0000","last_change":"2024-08-17T02:45:20.044663+0000","last_active":"2024-08-17T02:45:20.048440+0000","last_peered":"2024-08-17T02:45:20.048440+0000","last_clean":"2024-08-17T02:45:20.048440+0000","last_became_active":"2024-08-17T02:45:20.044412+0000","last_became_peered":"2024-08-17T02:45:20.044412+0000","last_unstale":"2024-08-17T02:45:20.048440+0000","last_undegraded":"2024-08-17T02:45:20.048440+0000","last_fullsized":"2024-08-17T02:45:20.048440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:36:50.133729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.038551+0000","last_change":"2024-08-17T02:45:20.035833+0000","last_active":"2024-08-17T02:45:20.038551+0000","last_peered":"2024-08-17T02:45:20.038551+0000","last_clean":"2024-08-17T02:45:20.038551+0000","last_became_active":"2024-08-17T02:45:20.035557+0000","last_became_peered":"2024-08-17T02:45:20.035557+0000","last_unstale":"2024-08-17T02:45:20.038551+0000","last_undegraded":"2024-08-17T02:45:20.038551+0000","last_fullsized":"2024-08-17T02:45:20.038551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:16:04.128578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.031258+0000","last_change":"2024-08-17T02:45:20.027132+0000","last_active":"2024-08-17T02:45:20.031258+0000","last_peered":"2024-08-17T02:45:20.031258+0000","last_clean":"2024-08-17T02:45:20.031258+0000","last_became_active":"2024-08-17T02:45:20.026869+0000","last_became_peered":"2024-08-17T02:45:20.026869+0000","last_unstale":"2024-08-17T02:45:20.031258+0000","last_undegraded":"2024-08-17T02:45:20.031258+0000","last_fullsized":"2024-08-17T02:45:20.031258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:50:24.600922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.050217+0000","last_change":"2024-08-17T02:45:20.042821+0000","last_active":"2024-08-17T02:45:20.050217+0000","last_peered":"2024-08-17T02:45:20.050217+0000","last_clean":"2024-08-17T02:45:20.050217+0000","last_became_active":"2024-08-17T02:45:20.042556+0000","last_became_peered":"2024-08-17T02:45:20.042556+0000","last_unstale":"2024-08-17T02:45:20.050217+0000","last_undegraded":"2024-08-17T02:45:20.050217+0000","last_fullsized":"2024-08-17T02:45:20.050217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:04:44.206575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.034352+0000","last_change":"2024-08-17T02:45:20.030193+0000","last_active":"2024-08-17T02:45:20.034352+0000","last_peered":"2024-08-17T02:45:20.034352+0000","last_clean":"2024-08-17T02:45:20.034352+0000","last_became_active":"2024-08-17T02:45:20.029651+0000","last_became_peered":"2024-08-17T02:45:20.029651+0000","last_unstale":"2024-08-17T02:45:20.034352+0000","last_undegraded":"2024-08-17T02:45:20.034352+0000","last_fullsized":"2024-08-17T02:45:20.034352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:09:19.783891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047891+0000","last_change":"2024-08-17T02:45:20.040335+0000","last_active":"2024-08-17T02:45:20.047891+0000","last_peered":"2024-08-17T02:45:20.047891+0000","last_clean":"2024-08-17T02:45:20.047891+0000","last_became_active":"2024-08-17T02:45:20.039917+0000","last_became_peered":"2024-08-17T02:45:20.039917+0000","last_unstale":"2024-08-17T02:45:20.047891+0000","last_undegraded":"2024-08-17T02:45:20.047891+0000","last_fullsized":"2024-08-17T02:45:20.047891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:08.644038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047626+0000","last_change":"2024-08-17T02:45:20.036503+0000","last_active":"2024-08-17T02:45:20.047626+0000","last_peered":"2024-08-17T02:45:20.047626+0000","last_clean":"2024-08-17T02:45:20.047626+0000","last_became_active":"2024-08-17T02:45:20.033814+0000","last_became_peered":"2024-08-17T02:45:20.033814+0000","last_unstale":"2024-08-17T02:45:20.047626+0000","last_undegraded":"2024-08-17T02:45:20.047626+0000","last_fullsized":"2024-08-17T02:45:20.047626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:32:46.909589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051312+0000","last_change":"2024-08-17T02:45:20.046161+0000","last_active":"2024-08-17T02:45:20.051312+0000","last_peered":"2024-08-17T02:45:20.051312+0000","last_clean":"2024-08-17T02:45:20.051312+0000","last_became_active":"2024-08-17T02:45:20.042204+0000","last_became_peered":"2024-08-17T02:45:20.042204+0000","last_unstale":"2024-08-17T02:45:20.051312+0000","last_undegraded":"2024-08-17T02:45:20.051312+0000","last_fullsized":"2024-08-17T02:45:20.051312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:32:46.909589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.021896+0000","last_change":"2024-08-17T02:45:20.019114+0000","last_active":"2024-08-17T02:45:20.021896+0000","last_peered":"2024-08-17T02:45:20.021896+0000","last_clean":"2024-08-17T02:45:20.021896+0000","last_became_active":"2024-08-17T02:45:20.018864+0000","last_became_peered":"2024-08-17T02:45:20.018864+0000","last_unstale":"2024-08-17T02:45:20.021896+0000","last_undegraded":"2024-08-17T02:45:20.021896+0000","last_fullsized":"2024-08-17T02:45:20.021896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:05:58.485958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048643+0000","last_change":"2024-08-17T02:45:20.044934+0000","last_active":"2024-08-17T02:45:20.048643+0000","last_peered":"2024-08-17T02:45:20.048643+0000","last_clean":"2024-08-17T02:45:20.048643+0000","last_became_active":"2024-08-17T02:45:20.044408+0000","last_became_peered":"2024-08-17T02:45:20.044408+0000","last_unstale":"2024-08-17T02:45:20.048643+0000","last_undegraded":"2024-08-17T02:45:20.048643+0000","last_fullsized":"2024-08-17T02:45:20.048643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:50:24.600922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.049471+0000","last_change":"2024-08-17T02:45:20.044628+0000","last_active":"2024-08-17T02:45:20.049471+0000","last_peered":"2024-08-17T02:45:20.049471+0000","last_clean":"2024-08-17T02:45:20.049471+0000","last_became_active":"2024-08-17T02:45:20.044329+0000","last_became_peered":"2024-08-17T02:45:20.044329+0000","last_unstale":"2024-08-17T02:45:20.049471+0000","last_undegraded":"2024-08-17T02:45:20.049471+0000","last_fullsized":"2024-08-17T02:45:20.049471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:31.312177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048512+0000","last_change":"2024-08-17T02:45:20.042172+0000","last_active":"2024-08-17T02:45:20.048512+0000","last_peered":"2024-08-17T02:45:20.048512+0000","last_clean":"2024-08-17T02:45:20.048512+0000","last_became_active":"2024-08-17T02:45:20.041703+0000","last_became_peered":"2024-08-17T02:45:20.041703+0000","last_unstale":"2024-08-17T02:45:20.048512+0000","last_undegraded":"2024-08-17T02:45:20.048512+0000","last_fullsized":"2024-08-17T02:45:20.048512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:42.062952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052573+0000","last_change":"2024-08-17T02:45:20.045385+0000","last_active":"2024-08-17T02:45:20.052573+0000","last_peered":"2024-08-17T02:45:20.052573+0000","last_clean":"2024-08-17T02:45:20.052573+0000","last_became_active":"2024-08-17T02:45:20.045105+0000","last_became_peered":"2024-08-17T02:45:20.045105+0000","last_unstale":"2024-08-17T02:45:20.052573+0000","last_undegraded":"2024-08-17T02:45:20.052573+0000","last_fullsized":"2024-08-17T02:45:20.052573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:33:18.470465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051658+0000","last_change":"2024-08-17T02:45:20.039999+0000","last_active":"2024-08-17T02:45:20.051658+0000","last_peered":"2024-08-17T02:45:20.051658+0000","last_clean":"2024-08-17T02:45:20.051658+0000","last_became_active":"2024-08-17T02:45:20.039656+0000","last_became_peered":"2024-08-17T02:45:20.039656+0000","last_unstale":"2024-08-17T02:45:20.051658+0000","last_undegraded":"2024-08-17T02:45:20.051658+0000","last_fullsized":"2024-08-17T02:45:20.051658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:02:17.358927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.049636+0000","last_change":"2024-08-17T02:45:20.042360+0000","last_active":"2024-08-17T02:45:20.049636+0000","last_peered":"2024-08-17T02:45:20.049636+0000","last_clean":"2024-08-17T02:45:20.049636+0000","last_became_active":"2024-08-17T02:45:20.042078+0000","last_became_peered":"2024-08-17T02:45:20.042078+0000","last_unstale":"2024-08-17T02:45:20.049636+0000","last_undegraded":"2024-08-17T02:45:20.049636+0000","last_fullsized":"2024-08-17T02:45:20.049636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:41:49.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040453+0000","last_change":"2024-08-17T02:45:20.026340+0000","last_active":"2024-08-17T02:45:20.040453+0000","last_peered":"2024-08-17T02:45:20.040453+0000","last_clean":"2024-08-17T02:45:20.040453+0000","last_became_active":"2024-08-17T02:45:20.026045+0000","last_became_peered":"2024-08-17T02:45:20.026045+0000","last_unstale":"2024-08-17T02:45:20.040453+0000","last_undegraded":"2024-08-17T02:45:20.040453+0000","last_fullsized":"2024-08-17T02:45:20.040453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:15:44.912061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052119+0000","last_change":"2024-08-17T02:45:20.037915+0000","last_active":"2024-08-17T02:45:20.052119+0000","last_peered":"2024-08-17T02:45:20.052119+0000","last_clean":"2024-08-17T02:45:20.052119+0000","last_became_active":"2024-08-17T02:45:20.034495+0000","last_became_peered":"2024-08-17T02:45:20.034495+0000","last_unstale":"2024-08-17T02:45:20.052119+0000","last_undegraded":"2024-08-17T02:45:20.052119+0000","last_fullsized":"2024-08-17T02:45:20.052119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:37:11.925388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.056304+0000","last_change":"2024-08-17T02:45:20.054620+0000","last_active":"2024-08-17T02:45:20.056304+0000","last_peered":"2024-08-17T02:45:20.056304+0000","last_clean":"2024-08-17T02:45:20.056304+0000","last_became_active":"2024-08-17T02:45:20.054100+0000","last_became_peered":"2024-08-17T02:45:20.054100+0000","last_unstale":"2024-08-17T02:45:20.056304+0000","last_undegraded":"2024-08-17T02:45:20.056304+0000","last_fullsized":"2024-08-17T02:45:20.056304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:08.644038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.041022+0000","last_change":"2024-08-17T02:45:20.031250+0000","last_active":"2024-08-17T02:45:20.041022+0000","last_peered":"2024-08-17T02:45:20.041022+0000","last_clean":"2024-08-17T02:45:20.041022+0000","last_became_active":"2024-08-17T02:45:20.026261+0000","last_became_peered":"2024-08-17T02:45:20.026261+0000","last_unstale":"2024-08-17T02:45:20.041022+0000","last_undegraded":"2024-08-17T02:45:20.041022+0000","last_fullsized":"2024-08-17T02:45:20.041022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:54:59.143806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048503+0000","last_change":"2024-08-17T02:45:20.041999+0000","last_active":"2024-08-17T02:45:20.048503+0000","last_peered":"2024-08-17T02:45:20.048503+0000","last_clean":"2024-08-17T02:45:20.048503+0000","last_became_active":"2024-08-17T02:45:20.041438+0000","last_became_peered":"2024-08-17T02:45:20.041438+0000","last_unstale":"2024-08-17T02:45:20.048503+0000","last_undegraded":"2024-08-17T02:45:20.048503+0000","last_fullsized":"2024-08-17T02:45:20.048503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:20.121506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.032037+0000","last_change":"2024-08-17T02:45:20.027927+0000","last_active":"2024-08-17T02:45:20.032037+0000","last_peered":"2024-08-17T02:45:20.032037+0000","last_clean":"2024-08-17T02:45:20.032037+0000","last_became_active":"2024-08-17T02:45:20.026932+0000","last_became_peered":"2024-08-17T02:45:20.026932+0000","last_unstale":"2024-08-17T02:45:20.032037+0000","last_undegraded":"2024-08-17T02:45:20.032037+0000","last_fullsized":"2024-08-17T02:45:20.032037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:33:12.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.032654+0000","last_change":"2024-08-17T02:45:20.031118+0000","last_active":"2024-08-17T02:45:20.032654+0000","last_peered":"2024-08-17T02:45:20.032654+0000","last_clean":"2024-08-17T02:45:20.032654+0000","last_became_active":"2024-08-17T02:45:20.030216+0000","last_became_peered":"2024-08-17T02:45:20.030216+0000","last_unstale":"2024-08-17T02:45:20.032654+0000","last_undegraded":"2024-08-17T02:45:20.032654+0000","last_fullsized":"2024-08-17T02:45:20.032654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:41:33.774280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051361+0000","last_change":"2024-08-17T02:45:20.045808+0000","last_active":"2024-08-17T02:45:20.051361+0000","last_peered":"2024-08-17T02:45:20.051361+0000","last_clean":"2024-08-17T02:45:20.051361+0000","last_became_active":"2024-08-17T02:45:20.045029+0000","last_became_peered":"2024-08-17T02:45:20.045029+0000","last_unstale":"2024-08-17T02:45:20.051361+0000","last_undegraded":"2024-08-17T02:45:20.051361+0000","last_fullsized":"2024-08-17T02:45:20.051361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:30.359495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040472+0000","last_change":"2024-08-17T02:45:20.028434+0000","last_active":"2024-08-17T02:45:20.040472+0000","last_peered":"2024-08-17T02:45:20.040472+0000","last_clean":"2024-08-17T02:45:20.040472+0000","last_became_active":"2024-08-17T02:45:20.026769+0000","last_became_peered":"2024-08-17T02:45:20.026769+0000","last_unstale":"2024-08-17T02:45:20.040472+0000","last_undegraded":"2024-08-17T02:45:20.040472+0000","last_fullsized":"2024-08-17T02:45:20.040472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:40:57.163688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051347+0000","last_change":"2024-08-17T02:45:20.046140+0000","last_active":"2024-08-17T02:45:20.051347+0000","last_peered":"2024-08-17T02:45:20.051347+0000","last_clean":"2024-08-17T02:45:20.051347+0000","last_became_active":"2024-08-17T02:45:20.043780+0000","last_became_peered":"2024-08-17T02:45:20.043780+0000","last_unstale":"2024-08-17T02:45:20.051347+0000","last_undegraded":"2024-08-17T02:45:20.051347+0000","last_fullsized":"2024-08-17T02:45:20.051347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:15:44.912061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047741+0000","last_change":"2024-08-17T02:45:20.039484+0000","last_active":"2024-08-17T02:45:20.047741+0000","last_peered":"2024-08-17T02:45:20.047741+0000","last_clean":"2024-08-17T02:45:20.047741+0000","last_became_active":"2024-08-17T02:45:20.039202+0000","last_became_peered":"2024-08-17T02:45:20.039202+0000","last_unstale":"2024-08-17T02:45:20.047741+0000","last_undegraded":"2024-08-17T02:45:20.047741+0000","last_fullsized":"2024-08-17T02:45:20.047741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:37:57.900723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051555+0000","last_change":"2024-08-17T02:45:20.039636+0000","last_active":"2024-08-17T02:45:20.051555+0000","last_peered":"2024-08-17T02:45:20.051555+0000","last_clean":"2024-08-17T02:45:20.051555+0000","last_became_active":"2024-08-17T02:45:20.039357+0000","last_became_peered":"2024-08-17T02:45:20.039357+0000","last_unstale":"2024-08-17T02:45:20.051555+0000","last_undegraded":"2024-08-17T02:45:20.051555+0000","last_fullsized":"2024-08-17T02:45:20.051555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:53:40.660806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052416+0000","last_change":"2024-08-17T02:45:20.044576+0000","last_active":"2024-08-17T02:45:20.052416+0000","last_peered":"2024-08-17T02:45:20.052416+0000","last_clean":"2024-08-17T02:45:20.052416+0000","last_became_active":"2024-08-17T02:45:20.044338+0000","last_became_peered":"2024-08-17T02:45:20.044338+0000","last_unstale":"2024-08-17T02:45:20.052416+0000","last_undegraded":"2024-08-17T02:45:20.052416+0000","last_fullsized":"2024-08-17T02:45:20.052416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:33:12.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.049261+0000","last_change":"2024-08-17T02:45:20.041424+0000","last_active":"2024-08-17T02:45:20.049261+0000","last_peered":"2024-08-17T02:45:20.049261+0000","last_clean":"2024-08-17T02:45:20.049261+0000","last_became_active":"2024-08-17T02:45:20.041158+0000","last_became_peered":"2024-08-17T02:45:20.041158+0000","last_unstale":"2024-08-17T02:45:20.049261+0000","last_undegraded":"2024-08-17T02:45:20.049261+0000","last_fullsized":"2024-08-17T02:45:20.049261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:09:19.783891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.041640+0000","last_change":"2024-08-17T02:45:20.027101+0000","last_active":"2024-08-17T02:45:20.041640+0000","last_peered":"2024-08-17T02:45:20.041640+0000","last_clean":"2024-08-17T02:45:20.041640+0000","last_became_active":"2024-08-17T02:45:20.026792+0000","last_became_peered":"2024-08-17T02:45:20.026792+0000","last_unstale":"2024-08-17T02:45:20.041640+0000","last_undegraded":"2024-08-17T02:45:20.041640+0000","last_fullsized":"2024-08-17T02:45:20.041640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:27:01.489949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047767+0000","last_change":"2024-08-17T02:45:20.044023+0000","last_active":"2024-08-17T02:45:20.047767+0000","last_peered":"2024-08-17T02:45:20.047767+0000","last_clean":"2024-08-17T02:45:20.047767+0000","last_became_active":"2024-08-17T02:45:20.043731+0000","last_became_peered":"2024-08-17T02:45:20.043731+0000","last_unstale":"2024-08-17T02:45:20.047767+0000","last_undegraded":"2024-08-17T02:45:20.047767+0000","last_fullsized":"2024-08-17T02:45:20.047767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:41:33.774280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048891+0000","last_change":"2024-08-17T02:45:20.042822+0000","last_active":"2024-08-17T02:45:20.048891+0000","last_peered":"2024-08-17T02:45:20.048891+0000","last_clean":"2024-08-17T02:45:20.048891+0000","last_became_active":"2024-08-17T02:45:20.042501+0000","last_became_peered":"2024-08-17T02:45:20.042501+0000","last_unstale":"2024-08-17T02:45:20.048891+0000","last_undegraded":"2024-08-17T02:45:20.048891+0000","last_fullsized":"2024-08-17T02:45:20.048891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:53:40.660806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040933+0000","last_change":"2024-08-17T02:45:20.032549+0000","last_active":"2024-08-17T02:45:20.040933+0000","last_peered":"2024-08-17T02:45:20.040933+0000","last_clean":"2024-08-17T02:45:20.040933+0000","last_became_active":"2024-08-17T02:45:20.032196+0000","last_became_peered":"2024-08-17T02:45:20.032196+0000","last_unstale":"2024-08-17T02:45:20.040933+0000","last_undegraded":"2024-08-17T02:45:20.040933+0000","last_fullsized":"2024-08-17T02:45:20.040933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:08.644038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.053812+0000","last_change":"2024-08-17T02:45:19.040955+0000","last_active":"2024-08-17T02:45:20.053812+0000","last_peered":"2024-08-17T02:45:20.053812+0000","last_clean":"2024-08-17T02:45:20.053812+0000","last_became_active":"2024-08-17T02:45:19.039340+0000","last_became_peered":"2024-08-17T02:45:19.039340+0000","last_unstale":"2024-08-17T02:45:20.053812+0000","last_undegraded":"2024-08-17T02:45:20.053812+0000","last_fullsized":"2024-08-17T02:45:20.053812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040815+0000","last_change":"2024-08-17T02:45:20.035907+0000","last_active":"2024-08-17T02:45:20.040815+0000","last_peered":"2024-08-17T02:45:20.040815+0000","last_clean":"2024-08-17T02:45:20.040815+0000","last_became_active":"2024-08-17T02:45:20.035670+0000","last_became_peered":"2024-08-17T02:45:20.035670+0000","last_unstale":"2024-08-17T02:45:20.040815+0000","last_undegraded":"2024-08-17T02:45:20.040815+0000","last_fullsized":"2024-08-17T02:45:20.040815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:41:49.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054523+0000","last_change":"2024-08-17T02:45:19.048167+0000","last_active":"2024-08-17T02:45:20.054523+0000","last_peered":"2024-08-17T02:45:20.054523+0000","last_clean":"2024-08-17T02:45:20.054523+0000","last_became_active":"2024-08-17T02:45:19.047739+0000","last_became_peered":"2024-08-17T02:45:19.047739+0000","last_unstale":"2024-08-17T02:45:20.054523+0000","last_undegraded":"2024-08-17T02:45:20.054523+0000","last_fullsized":"2024-08-17T02:45:20.054523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:33:17.468626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.033343+0000","last_change":"2024-08-17T02:45:19.043960+0000","last_active":"2024-08-17T02:45:20.033343+0000","last_peered":"2024-08-17T02:45:20.033343+0000","last_clean":"2024-08-17T02:45:20.033343+0000","last_became_active":"2024-08-17T02:45:19.042684+0000","last_became_peered":"2024-08-17T02:45:19.042684+0000","last_unstale":"2024-08-17T02:45:20.033343+0000","last_undegraded":"2024-08-17T02:45:20.033343+0000","last_fullsized":"2024-08-17T02:45:20.033343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.038584+0000","last_change":"2024-08-17T02:45:20.034766+0000","last_active":"2024-08-17T02:45:20.038584+0000","last_peered":"2024-08-17T02:45:20.038584+0000","last_clean":"2024-08-17T02:45:20.038584+0000","last_became_active":"2024-08-17T02:45:20.034450+0000","last_became_peered":"2024-08-17T02:45:20.034450+0000","last_unstale":"2024-08-17T02:45:20.038584+0000","last_undegraded":"2024-08-17T02:45:20.038584+0000","last_fullsized":"2024-08-17T02:45:20.038584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:02:55.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.013694+0000","last_change":"2024-08-17T02:45:19.050133+0000","last_active":"2024-08-17T02:45:20.013694+0000","last_peered":"2024-08-17T02:45:20.013694+0000","last_clean":"2024-08-17T02:45:20.013694+0000","last_became_active":"2024-08-17T02:45:19.049873+0000","last_became_peered":"2024-08-17T02:45:19.049873+0000","last_unstale":"2024-08-17T02:45:20.013694+0000","last_undegraded":"2024-08-17T02:45:20.013694+0000","last_fullsized":"2024-08-17T02:45:20.013694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:05:57.484119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.031101+0000","last_change":"2024-08-17T02:45:20.023134+0000","last_active":"2024-08-17T02:45:20.031101+0000","last_peered":"2024-08-17T02:45:20.031101+0000","last_clean":"2024-08-17T02:45:20.031101+0000","last_became_active":"2024-08-17T02:45:20.022844+0000","last_became_peered":"2024-08-17T02:45:20.022844+0000","last_unstale":"2024-08-17T02:45:20.031101+0000","last_undegraded":"2024-08-17T02:45:20.031101+0000","last_fullsized":"2024-08-17T02:45:20.031101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:30.359495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.018045+0000","last_change":"2024-08-17T02:45:19.046922+0000","last_active":"2024-08-17T02:45:20.018045+0000","last_peered":"2024-08-17T02:45:20.018045+0000","last_clean":"2024-08-17T02:45:20.018045+0000","last_became_active":"2024-08-17T02:45:19.046578+0000","last_became_peered":"2024-08-17T02:45:19.046578+0000","last_unstale":"2024-08-17T02:45:20.018045+0000","last_undegraded":"2024-08-17T02:45:20.018045+0000","last_fullsized":"2024-08-17T02:45:20.018045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047730+0000","last_change":"2024-08-17T02:45:20.039463+0000","last_active":"2024-08-17T02:45:20.047730+0000","last_peered":"2024-08-17T02:45:20.047730+0000","last_clean":"2024-08-17T02:45:20.047730+0000","last_became_active":"2024-08-17T02:45:20.038869+0000","last_became_peered":"2024-08-17T02:45:20.038869+0000","last_unstale":"2024-08-17T02:45:20.047730+0000","last_undegraded":"2024-08-17T02:45:20.047730+0000","last_fullsized":"2024-08-17T02:45:20.047730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:33:18.470465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.035076+0000","last_change":"2024-08-17T02:45:19.048740+0000","last_active":"2024-08-17T02:45:20.035076+0000","last_peered":"2024-08-17T02:45:20.035076+0000","last_clean":"2024-08-17T02:45:20.035076+0000","last_became_active":"2024-08-17T02:45:19.047688+0000","last_became_peered":"2024-08-17T02:45:19.047688+0000","last_unstale":"2024-08-17T02:45:20.035076+0000","last_undegraded":"2024-08-17T02:45:20.035076+0000","last_fullsized":"2024-08-17T02:45:20.035076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:43.862231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051327+0000","last_change":"2024-08-17T02:45:20.046160+0000","last_active":"2024-08-17T02:45:20.051327+0000","last_peered":"2024-08-17T02:45:20.051327+0000","last_clean":"2024-08-17T02:45:20.051327+0000","last_became_active":"2024-08-17T02:45:20.045893+0000","last_became_peered":"2024-08-17T02:45:20.045893+0000","last_unstale":"2024-08-17T02:45:20.051327+0000","last_undegraded":"2024-08-17T02:45:20.051327+0000","last_fullsized":"2024-08-17T02:45:20.051327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:03:07.944484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054309+0000","last_change":"2024-08-17T02:45:19.043386+0000","last_active":"2024-08-17T02:45:20.054309+0000","last_peered":"2024-08-17T02:45:20.054309+0000","last_clean":"2024-08-17T02:45:20.054309+0000","last_became_active":"2024-08-17T02:45:19.043050+0000","last_became_peered":"2024-08-17T02:45:19.043050+0000","last_unstale":"2024-08-17T02:45:20.054309+0000","last_undegraded":"2024-08-17T02:45:20.054309+0000","last_fullsized":"2024-08-17T02:45:20.054309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.042043+0000","last_change":"2024-08-17T02:45:20.029004+0000","last_active":"2024-08-17T02:45:20.042043+0000","last_peered":"2024-08-17T02:45:20.042043+0000","last_clean":"2024-08-17T02:45:20.042043+0000","last_became_active":"2024-08-17T02:45:20.028698+0000","last_became_peered":"2024-08-17T02:45:20.028698+0000","last_unstale":"2024-08-17T02:45:20.042043+0000","last_undegraded":"2024-08-17T02:45:20.042043+0000","last_fullsized":"2024-08-17T02:45:20.042043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:42.062952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.058174+0000","last_change":"2024-08-17T02:45:19.034956+0000","last_active":"2024-08-17T02:45:20.058174+0000","last_peered":"2024-08-17T02:45:20.058174+0000","last_clean":"2024-08-17T02:45:20.058174+0000","last_became_active":"2024-08-17T02:45:19.034658+0000","last_became_peered":"2024-08-17T02:45:19.034658+0000","last_unstale":"2024-08-17T02:45:20.058174+0000","last_undegraded":"2024-08-17T02:45:20.058174+0000","last_fullsized":"2024-08-17T02:45:20.058174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048217+0000","last_change":"2024-08-17T02:45:20.039165+0000","last_active":"2024-08-17T02:45:20.048217+0000","last_peered":"2024-08-17T02:45:20.048217+0000","last_clean":"2024-08-17T02:45:20.048217+0000","last_became_active":"2024-08-17T02:45:20.038923+0000","last_became_peered":"2024-08-17T02:45:20.038923+0000","last_unstale":"2024-08-17T02:45:20.048217+0000","last_undegraded":"2024-08-17T02:45:20.048217+0000","last_fullsized":"2024-08-17T02:45:20.048217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:03:07.944484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.053604+0000","last_change":"2024-08-17T02:45:19.044788+0000","last_active":"2024-08-17T02:45:20.053604+0000","last_peered":"2024-08-17T02:45:20.053604+0000","last_clean":"2024-08-17T02:45:20.053604+0000","last_became_active":"2024-08-17T02:45:19.044538+0000","last_became_peered":"2024-08-17T02:45:19.044538+0000","last_unstale":"2024-08-17T02:45:20.053604+0000","last_undegraded":"2024-08-17T02:45:20.053604+0000","last_fullsized":"2024-08-17T02:45:20.053604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:44:52.129209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052358+0000","last_change":"2024-08-17T02:45:20.043190+0000","last_active":"2024-08-17T02:45:20.052358+0000","last_peered":"2024-08-17T02:45:20.052358+0000","last_clean":"2024-08-17T02:45:20.052358+0000","last_became_active":"2024-08-17T02:45:20.042860+0000","last_became_peered":"2024-08-17T02:45:20.042860+0000","last_unstale":"2024-08-17T02:45:20.052358+0000","last_undegraded":"2024-08-17T02:45:20.052358+0000","last_fullsized":"2024-08-17T02:45:20.052358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:55:18.896035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.013972+0000","last_change":"2024-08-17T02:45:19.047294+0000","last_active":"2024-08-17T02:45:20.013972+0000","last_peered":"2024-08-17T02:45:20.013972+0000","last_clean":"2024-08-17T02:45:20.013972+0000","last_became_active":"2024-08-17T02:45:19.042180+0000","last_became_peered":"2024-08-17T02:45:19.042180+0000","last_unstale":"2024-08-17T02:45:20.013972+0000","last_undegraded":"2024-08-17T02:45:20.013972+0000","last_fullsized":"2024-08-17T02:45:20.013972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:37:56.898883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052410+0000","last_change":"2024-08-17T02:45:20.038226+0000","last_active":"2024-08-17T02:45:20.052410+0000","last_peered":"2024-08-17T02:45:20.052410+0000","last_clean":"2024-08-17T02:45:20.052410+0000","last_became_active":"2024-08-17T02:45:20.034659+0000","last_became_peered":"2024-08-17T02:45:20.034659+0000","last_unstale":"2024-08-17T02:45:20.052410+0000","last_undegraded":"2024-08-17T02:45:20.052410+0000","last_fullsized":"2024-08-17T02:45:20.052410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:27:53.329893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.036680+0000","last_change":"2024-08-17T02:45:19.052673+0000","last_active":"2024-08-17T02:45:20.036680+0000","last_peered":"2024-08-17T02:45:20.036680+0000","last_clean":"2024-08-17T02:45:20.036680+0000","last_became_active":"2024-08-17T02:45:19.052404+0000","last_became_peered":"2024-08-17T02:45:19.052404+0000","last_unstale":"2024-08-17T02:45:20.036680+0000","last_undegraded":"2024-08-17T02:45:20.036680+0000","last_fullsized":"2024-08-17T02:45:20.036680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:46:09.850664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.039882+0000","last_change":"2024-08-17T02:45:20.031377+0000","last_active":"2024-08-17T02:45:20.039882+0000","last_peered":"2024-08-17T02:45:20.039882+0000","last_clean":"2024-08-17T02:45:20.039882+0000","last_became_active":"2024-08-17T02:45:20.031103+0000","last_became_peered":"2024-08-17T02:45:20.031103+0000","last_unstale":"2024-08-17T02:45:20.039882+0000","last_undegraded":"2024-08-17T02:45:20.039882+0000","last_fullsized":"2024-08-17T02:45:20.039882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:04:44.206575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054426+0000","last_change":"2024-08-17T02:45:19.042568+0000","last_active":"2024-08-17T02:45:20.054426+0000","last_peered":"2024-08-17T02:45:20.054426+0000","last_clean":"2024-08-17T02:45:20.054426+0000","last_became_active":"2024-08-17T02:45:19.041846+0000","last_became_peered":"2024-08-17T02:45:19.041846+0000","last_unstale":"2024-08-17T02:45:20.054426+0000","last_undegraded":"2024-08-17T02:45:20.054426+0000","last_fullsized":"2024-08-17T02:45:20.054426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.042968+0000","last_change":"2024-08-17T02:45:20.029516+0000","last_active":"2024-08-17T02:45:20.042968+0000","last_peered":"2024-08-17T02:45:20.042968+0000","last_clean":"2024-08-17T02:45:20.042968+0000","last_became_active":"2024-08-17T02:45:20.027167+0000","last_became_peered":"2024-08-17T02:45:20.027167+0000","last_unstale":"2024-08-17T02:45:20.042968+0000","last_undegraded":"2024-08-17T02:45:20.042968+0000","last_fullsized":"2024-08-17T02:45:20.042968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:20.121506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.056082+0000","last_change":"2024-08-17T02:45:19.038843+0000","last_active":"2024-08-17T02:45:20.056082+0000","last_peered":"2024-08-17T02:45:20.056082+0000","last_clean":"2024-08-17T02:45:20.056082+0000","last_became_active":"2024-08-17T02:45:19.038608+0000","last_became_peered":"2024-08-17T02:45:19.038608+0000","last_unstale":"2024-08-17T02:45:20.056082+0000","last_undegraded":"2024-08-17T02:45:20.056082+0000","last_fullsized":"2024-08-17T02:45:20.056082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:25:12.592566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047411+0000","last_change":"2024-08-17T02:45:20.037709+0000","last_active":"2024-08-17T02:45:20.047411+0000","last_peered":"2024-08-17T02:45:20.047411+0000","last_clean":"2024-08-17T02:45:20.047411+0000","last_became_active":"2024-08-17T02:45:20.037461+0000","last_became_peered":"2024-08-17T02:45:20.037461+0000","last_unstale":"2024-08-17T02:45:20.047411+0000","last_undegraded":"2024-08-17T02:45:20.047411+0000","last_fullsized":"2024-08-17T02:45:20.047411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:05:58.485958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051949+0000","last_change":"2024-08-17T02:45:19.039018+0000","last_active":"2024-08-17T02:45:20.051949+0000","last_peered":"2024-08-17T02:45:20.051949+0000","last_clean":"2024-08-17T02:45:20.051949+0000","last_became_active":"2024-08-17T02:45:19.036850+0000","last_became_peered":"2024-08-17T02:45:19.036850+0000","last_unstale":"2024-08-17T02:45:20.051949+0000","last_undegraded":"2024-08-17T02:45:20.051949+0000","last_fullsized":"2024-08-17T02:45:20.051949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.033842+0000","last_change":"2024-08-17T02:45:20.030655+0000","last_active":"2024-08-17T02:45:20.033842+0000","last_peered":"2024-08-17T02:45:20.033842+0000","last_clean":"2024-08-17T02:45:20.033842+0000","last_became_active":"2024-08-17T02:45:20.030439+0000","last_became_peered":"2024-08-17T02:45:20.030439+0000","last_unstale":"2024-08-17T02:45:20.033842+0000","last_undegraded":"2024-08-17T02:45:20.033842+0000","last_fullsized":"2024-08-17T02:45:20.033842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:15:44.912061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.055739+0000","last_change":"2024-08-17T02:45:19.045933+0000","last_active":"2024-08-17T02:45:20.055739+0000","last_peered":"2024-08-17T02:45:20.055739+0000","last_clean":"2024-08-17T02:45:20.055739+0000","last_became_active":"2024-08-17T02:45:19.045649+0000","last_became_peered":"2024-08-17T02:45:19.045649+0000","last_unstale":"2024-08-17T02:45:20.055739+0000","last_undegraded":"2024-08-17T02:45:20.055739+0000","last_fullsized":"2024-08-17T02:45:20.055739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048520+0000","last_change":"2024-08-17T02:45:20.043745+0000","last_active":"2024-08-17T02:45:20.048520+0000","last_peered":"2024-08-17T02:45:20.048520+0000","last_clean":"2024-08-17T02:45:20.048520+0000","last_became_active":"2024-08-17T02:45:20.043470+0000","last_became_peered":"2024-08-17T02:45:20.043470+0000","last_unstale":"2024-08-17T02:45:20.048520+0000","last_undegraded":"2024-08-17T02:45:20.048520+0000","last_fullsized":"2024-08-17T02:45:20.048520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:00:06.622218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.033701+0000","last_change":"2024-08-17T02:45:19.040219+0000","last_active":"2024-08-17T02:45:20.033701+0000","last_peered":"2024-08-17T02:45:20.033701+0000","last_clean":"2024-08-17T02:45:20.033701+0000","last_became_active":"2024-08-17T02:45:19.039816+0000","last_became_peered":"2024-08-17T02:45:19.039816+0000","last_unstale":"2024-08-17T02:45:20.033701+0000","last_undegraded":"2024-08-17T02:45:20.033701+0000","last_fullsized":"2024-08-17T02:45:20.033701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040997+0000","last_change":"2024-08-17T02:45:20.027155+0000","last_active":"2024-08-17T02:45:20.040997+0000","last_peered":"2024-08-17T02:45:20.040997+0000","last_clean":"2024-08-17T02:45:20.040997+0000","last_became_active":"2024-08-17T02:45:20.026102+0000","last_became_peered":"2024-08-17T02:45:20.026102+0000","last_unstale":"2024-08-17T02:45:20.040997+0000","last_undegraded":"2024-08-17T02:45:20.040997+0000","last_fullsized":"2024-08-17T02:45:20.040997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:05:05.162709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.017763+0000","last_change":"2024-08-17T02:45:16.001117+0000","last_active":"2024-08-17T02:45:20.017763+0000","last_peered":"2024-08-17T02:45:20.017763+0000","last_clean":"2024-08-17T02:45:20.017763+0000","last_became_active":"2024-08-17T02:45:16.000788+0000","last_became_peered":"2024-08-17T02:45:16.000788+0000","last_unstale":"2024-08-17T02:45:20.017763+0000","last_undegraded":"2024-08-17T02:45:20.017763+0000","last_fullsized":"2024-08-17T02:45:20.017763+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_clean_scrub_stamp":"2024-08-17T02:45:14.986707+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:27:37.295374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.014889+0000","last_change":"2024-08-17T02:45:19.042366+0000","last_active":"2024-08-17T02:45:20.014889+0000","last_peered":"2024-08-17T02:45:20.014889+0000","last_clean":"2024-08-17T02:45:20.014889+0000","last_became_active":"2024-08-17T02:45:19.039118+0000","last_became_peered":"2024-08-17T02:45:19.039118+0000","last_unstale":"2024-08-17T02:45:20.014889+0000","last_undegraded":"2024-08-17T02:45:20.014889+0000","last_fullsized":"2024-08-17T02:45:20.014889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:07.642199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.031922+0000","last_change":"2024-08-17T02:45:20.026830+0000","last_active":"2024-08-17T02:45:20.031922+0000","last_peered":"2024-08-17T02:45:20.031922+0000","last_clean":"2024-08-17T02:45:20.031922+0000","last_became_active":"2024-08-17T02:45:20.026135+0000","last_became_peered":"2024-08-17T02:45:20.026135+0000","last_unstale":"2024-08-17T02:45:20.031922+0000","last_undegraded":"2024-08-17T02:45:20.031922+0000","last_fullsized":"2024-08-17T02:45:20.031922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:37:57.900723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.013832+0000","last_change":"2024-08-17T02:45:19.046461+0000","last_active":"2024-08-17T02:45:20.013832+0000","last_peered":"2024-08-17T02:45:20.013832+0000","last_clean":"2024-08-17T02:45:20.013832+0000","last_became_active":"2024-08-17T02:45:19.042611+0000","last_became_peered":"2024-08-17T02:45:19.042611+0000","last_unstale":"2024-08-17T02:45:20.013832+0000","last_undegraded":"2024-08-17T02:45:20.013832+0000","last_fullsized":"2024-08-17T02:45:20.013832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:41:32.772441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052264+0000","last_change":"2024-08-17T02:45:20.043147+0000","last_active":"2024-08-17T02:45:20.052264+0000","last_peered":"2024-08-17T02:45:20.052264+0000","last_clean":"2024-08-17T02:45:20.052264+0000","last_became_active":"2024-08-17T02:45:20.042828+0000","last_became_peered":"2024-08-17T02:45:20.042828+0000","last_unstale":"2024-08-17T02:45:20.052264+0000","last_undegraded":"2024-08-17T02:45:20.052264+0000","last_fullsized":"2024-08-17T02:45:20.052264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:30.359495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.031941+0000","last_change":"2024-08-17T02:45:19.047594+0000","last_active":"2024-08-17T02:45:20.031941+0000","last_peered":"2024-08-17T02:45:20.031941+0000","last_clean":"2024-08-17T02:45:20.031941+0000","last_became_active":"2024-08-17T02:45:19.047271+0000","last_became_peered":"2024-08-17T02:45:19.047271+0000","last_unstale":"2024-08-17T02:45:20.031941+0000","last_undegraded":"2024-08-17T02:45:20.031941+0000","last_fullsized":"2024-08-17T02:45:20.031941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.050654+0000","last_change":"2024-08-17T02:45:20.043304+0000","last_active":"2024-08-17T02:45:20.050654+0000","last_peered":"2024-08-17T02:45:20.050654+0000","last_clean":"2024-08-17T02:45:20.050654+0000","last_became_active":"2024-08-17T02:45:20.038490+0000","last_became_peered":"2024-08-17T02:45:20.038490+0000","last_unstale":"2024-08-17T02:45:20.050654+0000","last_undegraded":"2024-08-17T02:45:20.050654+0000","last_fullsized":"2024-08-17T02:45:20.050654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:00:06.622218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.014144+0000","last_change":"2024-08-17T02:45:19.048638+0000","last_active":"2024-08-17T02:45:20.014144+0000","last_peered":"2024-08-17T02:45:20.014144+0000","last_clean":"2024-08-17T02:45:20.014144+0000","last_became_active":"2024-08-17T02:45:19.048346+0000","last_became_peered":"2024-08-17T02:45:19.048346+0000","last_unstale":"2024-08-17T02:45:20.014144+0000","last_undegraded":"2024-08-17T02:45:20.014144+0000","last_fullsized":"2024-08-17T02:45:20.014144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:46:09.850664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.034371+0000","last_change":"2024-08-17T02:45:20.030375+0000","last_active":"2024-08-17T02:45:20.034371+0000","last_peered":"2024-08-17T02:45:20.034371+0000","last_clean":"2024-08-17T02:45:20.034371+0000","last_became_active":"2024-08-17T02:45:20.029966+0000","last_became_peered":"2024-08-17T02:45:20.029966+0000","last_unstale":"2024-08-17T02:45:20.034371+0000","last_undegraded":"2024-08-17T02:45:20.034371+0000","last_fullsized":"2024-08-17T02:45:20.034371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:16:04.128578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.021330+0000","last_change":"2024-08-17T02:45:19.039495+0000","last_active":"2024-08-17T02:45:20.021330+0000","last_peered":"2024-08-17T02:45:20.021330+0000","last_clean":"2024-08-17T02:45:20.021330+0000","last_became_active":"2024-08-17T02:45:19.039218+0000","last_became_peered":"2024-08-17T02:45:19.039218+0000","last_unstale":"2024-08-17T02:45:20.021330+0000","last_undegraded":"2024-08-17T02:45:20.021330+0000","last_fullsized":"2024-08-17T02:45:20.021330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052935+0000","last_change":"2024-08-17T02:45:20.047183+0000","last_active":"2024-08-17T02:45:20.052935+0000","last_peered":"2024-08-17T02:45:20.052935+0000","last_clean":"2024-08-17T02:45:20.052935+0000","last_became_active":"2024-08-17T02:45:20.045627+0000","last_became_peered":"2024-08-17T02:45:20.045627+0000","last_unstale":"2024-08-17T02:45:20.052935+0000","last_undegraded":"2024-08-17T02:45:20.052935+0000","last_fullsized":"2024-08-17T02:45:20.052935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:02:17.358927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.016227+0000","last_change":"2024-08-17T02:45:19.043843+0000","last_active":"2024-08-17T02:45:20.016227+0000","last_peered":"2024-08-17T02:45:20.016227+0000","last_clean":"2024-08-17T02:45:20.016227+0000","last_became_active":"2024-08-17T02:45:19.043069+0000","last_became_peered":"2024-08-17T02:45:19.043069+0000","last_unstale":"2024-08-17T02:45:20.016227+0000","last_undegraded":"2024-08-17T02:45:20.016227+0000","last_fullsized":"2024-08-17T02:45:20.016227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:43.862231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051578+0000","last_change":"2024-08-17T02:45:20.038816+0000","last_active":"2024-08-17T02:45:20.051578+0000","last_peered":"2024-08-17T02:45:20.051578+0000","last_clean":"2024-08-17T02:45:20.051578+0000","last_became_active":"2024-08-17T02:45:20.038520+0000","last_became_peered":"2024-08-17T02:45:20.038520+0000","last_unstale":"2024-08-17T02:45:20.051578+0000","last_undegraded":"2024-08-17T02:45:20.051578+0000","last_fullsized":"2024-08-17T02:45:20.051578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:37:57.900723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:45:22.409624+0000","last_change":"2024-08-17T02:45:19.044921+0000","last_active":"2024-08-17T02:45:22.409624+0000","last_peered":"2024-08-17T02:45:22.409624+0000","last_clean":"2024-08-17T02:45:22.409624+0000","last_became_active":"2024-08-17T02:45:19.041305+0000","last_became_peered":"2024-08-17T02:45:19.041305+0000","last_unstale":"2024-08-17T02:45:22.409624+0000","last_undegraded":"2024-08-17T02:45:22.409624+0000","last_fullsized":"2024-08-17T02:45:22.409624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:03:27.020817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.041518+0000","last_change":"2024-08-17T02:45:20.035169+0000","last_active":"2024-08-17T02:45:20.041518+0000","last_peered":"2024-08-17T02:45:20.041518+0000","last_clean":"2024-08-17T02:45:20.041518+0000","last_became_active":"2024-08-17T02:45:20.034858+0000","last_became_peered":"2024-08-17T02:45:20.034858+0000","last_unstale":"2024-08-17T02:45:20.041518+0000","last_undegraded":"2024-08-17T02:45:20.041518+0000","last_fullsized":"2024-08-17T02:45:20.041518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:53:40.660806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054541+0000","last_change":"2024-08-17T02:45:19.049505+0000","last_active":"2024-08-17T02:45:20.054541+0000","last_peered":"2024-08-17T02:45:20.054541+0000","last_clean":"2024-08-17T02:45:20.054541+0000","last_became_active":"2024-08-17T02:45:19.048843+0000","last_became_peered":"2024-08-17T02:45:19.048843+0000","last_unstale":"2024-08-17T02:45:20.054541+0000","last_undegraded":"2024-08-17T02:45:20.054541+0000","last_fullsized":"2024-08-17T02:45:20.054541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:53:39.658967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.046988+0000","last_change":"2024-08-17T02:45:20.038861+0000","last_active":"2024-08-17T02:45:20.046988+0000","last_peered":"2024-08-17T02:45:20.046988+0000","last_clean":"2024-08-17T02:45:20.046988+0000","last_became_active":"2024-08-17T02:45:20.038597+0000","last_became_peered":"2024-08-17T02:45:20.038597+0000","last_unstale":"2024-08-17T02:45:20.046988+0000","last_undegraded":"2024-08-17T02:45:20.046988+0000","last_fullsized":"2024-08-17T02:45:20.046988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:02:55.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.034402+0000","last_change":"2024-08-17T02:45:19.035699+0000","last_active":"2024-08-17T02:45:20.034402+0000","last_peered":"2024-08-17T02:45:20.034402+0000","last_clean":"2024-08-17T02:45:20.034402+0000","last_became_active":"2024-08-17T02:45:19.035429+0000","last_became_peered":"2024-08-17T02:45:19.035429+0000","last_unstale":"2024-08-17T02:45:20.034402+0000","last_undegraded":"2024-08-17T02:45:20.034402+0000","last_fullsized":"2024-08-17T02:45:20.034402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:41:32.772441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.021682+0000","last_change":"2024-08-17T02:45:20.020728+0000","last_active":"2024-08-17T02:45:20.021682+0000","last_peered":"2024-08-17T02:45:20.021682+0000","last_clean":"2024-08-17T02:45:20.021682+0000","last_became_active":"2024-08-17T02:45:20.020439+0000","last_became_peered":"2024-08-17T02:45:20.020439+0000","last_unstale":"2024-08-17T02:45:20.021682+0000","last_undegraded":"2024-08-17T02:45:20.021682+0000","last_fullsized":"2024-08-17T02:45:20.021682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:32:46.909589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.016454+0000","last_change":"2024-08-17T02:45:19.044294+0000","last_active":"2024-08-17T02:45:20.016454+0000","last_peered":"2024-08-17T02:45:20.016454+0000","last_clean":"2024-08-17T02:45:20.016454+0000","last_became_active":"2024-08-17T02:45:19.044001+0000","last_became_peered":"2024-08-17T02:45:19.044001+0000","last_unstale":"2024-08-17T02:45:20.016454+0000","last_undegraded":"2024-08-17T02:45:20.016454+0000","last_fullsized":"2024-08-17T02:45:20.016454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:43.862231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047889+0000","last_change":"2024-08-17T02:45:20.040445+0000","last_active":"2024-08-17T02:45:20.047889+0000","last_peered":"2024-08-17T02:45:20.047889+0000","last_clean":"2024-08-17T02:45:20.047889+0000","last_became_active":"2024-08-17T02:45:20.040151+0000","last_became_peered":"2024-08-17T02:45:20.040151+0000","last_unstale":"2024-08-17T02:45:20.047889+0000","last_undegraded":"2024-08-17T02:45:20.047889+0000","last_fullsized":"2024-08-17T02:45:20.047889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:55:18.896035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.033393+0000","last_change":"2024-08-17T02:45:19.037364+0000","last_active":"2024-08-17T02:45:20.033393+0000","last_peered":"2024-08-17T02:45:20.033393+0000","last_clean":"2024-08-17T02:45:20.033393+0000","last_became_active":"2024-08-17T02:45:19.037020+0000","last_became_peered":"2024-08-17T02:45:19.037020+0000","last_unstale":"2024-08-17T02:45:20.033393+0000","last_undegraded":"2024-08-17T02:45:20.033393+0000","last_fullsized":"2024-08-17T02:45:20.033393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052107+0000","last_change":"2024-08-17T02:45:20.047671+0000","last_active":"2024-08-17T02:45:20.052107+0000","last_peered":"2024-08-17T02:45:20.052107+0000","last_clean":"2024-08-17T02:45:20.052107+0000","last_became_active":"2024-08-17T02:45:20.046596+0000","last_became_peered":"2024-08-17T02:45:20.046596+0000","last_unstale":"2024-08-17T02:45:20.052107+0000","last_undegraded":"2024-08-17T02:45:20.052107+0000","last_fullsized":"2024-08-17T02:45:20.052107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:46:10.852503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.057610+0000","last_change":"2024-08-17T02:45:19.040794+0000","last_active":"2024-08-17T02:45:20.057610+0000","last_peered":"2024-08-17T02:45:20.057610+0000","last_clean":"2024-08-17T02:45:20.057610+0000","last_became_active":"2024-08-17T02:45:19.038764+0000","last_became_peered":"2024-08-17T02:45:19.038764+0000","last_unstale":"2024-08-17T02:45:20.057610+0000","last_undegraded":"2024-08-17T02:45:20.057610+0000","last_fullsized":"2024-08-17T02:45:20.057610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:04:43.204736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.021022+0000","last_change":"2024-08-17T02:45:20.019301+0000","last_active":"2024-08-17T02:45:20.021022+0000","last_peered":"2024-08-17T02:45:20.021022+0000","last_clean":"2024-08-17T02:45:20.021022+0000","last_became_active":"2024-08-17T02:45:20.018641+0000","last_became_peered":"2024-08-17T02:45:20.018641+0000","last_unstale":"2024-08-17T02:45:20.021022+0000","last_undegraded":"2024-08-17T02:45:20.021022+0000","last_fullsized":"2024-08-17T02:45:20.021022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:47:06.908123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052207+0000","last_change":"2024-08-17T02:45:20.037807+0000","last_active":"2024-08-17T02:45:20.052207+0000","last_peered":"2024-08-17T02:45:20.052207+0000","last_clean":"2024-08-17T02:45:20.052207+0000","last_became_active":"2024-08-17T02:45:20.034485+0000","last_became_peered":"2024-08-17T02:45:20.034485+0000","last_unstale":"2024-08-17T02:45:20.052207+0000","last_undegraded":"2024-08-17T02:45:20.052207+0000","last_fullsized":"2024-08-17T02:45:20.052207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:29:54.794188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.055868+0000","last_change":"2024-08-17T02:45:19.046881+0000","last_active":"2024-08-17T02:45:20.055868+0000","last_peered":"2024-08-17T02:45:20.055868+0000","last_clean":"2024-08-17T02:45:20.055868+0000","last_became_active":"2024-08-17T02:45:19.046676+0000","last_became_peered":"2024-08-17T02:45:19.046676+0000","last_unstale":"2024-08-17T02:45:20.055868+0000","last_undegraded":"2024-08-17T02:45:20.055868+0000","last_fullsized":"2024-08-17T02:45:20.055868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.041726+0000","last_change":"2024-08-17T02:45:20.040519+0000","last_active":"2024-08-17T02:45:20.041726+0000","last_peered":"2024-08-17T02:45:20.041726+0000","last_clean":"2024-08-17T02:45:20.041726+0000","last_became_active":"2024-08-17T02:45:20.040246+0000","last_became_peered":"2024-08-17T02:45:20.040246+0000","last_unstale":"2024-08-17T02:45:20.041726+0000","last_undegraded":"2024-08-17T02:45:20.041726+0000","last_fullsized":"2024-08-17T02:45:20.041726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:46:10.852503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.055789+0000","last_change":"2024-08-17T02:45:19.045919+0000","last_active":"2024-08-17T02:45:20.055789+0000","last_peered":"2024-08-17T02:45:20.055789+0000","last_clean":"2024-08-17T02:45:20.055789+0000","last_became_active":"2024-08-17T02:45:19.045693+0000","last_became_peered":"2024-08-17T02:45:19.045693+0000","last_unstale":"2024-08-17T02:45:20.055789+0000","last_undegraded":"2024-08-17T02:45:20.055789+0000","last_fullsized":"2024-08-17T02:45:20.055789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:55:24.029624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.050681+0000","last_change":"2024-08-17T02:45:20.044015+0000","last_active":"2024-08-17T02:45:20.050681+0000","last_peered":"2024-08-17T02:45:20.050681+0000","last_clean":"2024-08-17T02:45:20.050681+0000","last_became_active":"2024-08-17T02:45:20.043471+0000","last_became_peered":"2024-08-17T02:45:20.043471+0000","last_unstale":"2024-08-17T02:45:20.050681+0000","last_undegraded":"2024-08-17T02:45:20.050681+0000","last_fullsized":"2024-08-17T02:45:20.050681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:20.121506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054182+0000","last_change":"2024-08-17T02:45:19.041823+0000","last_active":"2024-08-17T02:45:20.054182+0000","last_peered":"2024-08-17T02:45:20.054182+0000","last_clean":"2024-08-17T02:45:20.054182+0000","last_became_active":"2024-08-17T02:45:19.041481+0000","last_became_peered":"2024-08-17T02:45:19.041481+0000","last_unstale":"2024-08-17T02:45:20.054182+0000","last_undegraded":"2024-08-17T02:45:20.054182+0000","last_fullsized":"2024-08-17T02:45:20.054182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:03:27.020817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051425+0000","last_change":"2024-08-17T02:45:20.047339+0000","last_active":"2024-08-17T02:45:20.051425+0000","last_peered":"2024-08-17T02:45:20.051425+0000","last_clean":"2024-08-17T02:45:20.051425+0000","last_became_active":"2024-08-17T02:45:20.047111+0000","last_became_peered":"2024-08-17T02:45:20.047111+0000","last_unstale":"2024-08-17T02:45:20.051425+0000","last_undegraded":"2024-08-17T02:45:20.051425+0000","last_fullsized":"2024-08-17T02:45:20.051425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:46:59.247481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054521+0000","last_change":"2024-08-17T02:45:19.042331+0000","last_active":"2024-08-17T02:45:20.054521+0000","last_peered":"2024-08-17T02:45:20.054521+0000","last_clean":"2024-08-17T02:45:20.054521+0000","last_became_active":"2024-08-17T02:45:19.042033+0000","last_became_peered":"2024-08-17T02:45:19.042033+0000","last_unstale":"2024-08-17T02:45:20.054521+0000","last_undegraded":"2024-08-17T02:45:20.054521+0000","last_fullsized":"2024-08-17T02:45:20.054521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:25:12.592566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":75437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":217088,"data_stored":75313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298156,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073684,"statfs":{"total":96636764160,"available":96331452416,"internally_reserved":0,"allocated":819200,"data_stored":665773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":237568,"data_stored":77525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":221184,"data_stored":75883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301744,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070096,"statfs":{"total":96636764160,"available":96327778304,"internally_reserved":0,"allocated":229376,"data_stored":76865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":815104,"data_stored":665751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301740,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070100,"statfs":{"total":96636764160,"available":96327782400,"internally_reserved":0,"allocated":225280,"data_stored":75369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:45:29.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:45:29.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.466+0000 7fb3bc200640 1 Processor -- start 2024-08-17T02:45:29.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.466+0000 7fb3bc200640 1 -- start start 2024-08-17T02:45:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.467+0000 7fb3bc200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3b4155a20 0x7fb3b40adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.468+0000 7fb3bc200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b4156670 con 0x7fb3b406df00 2024-08-17T02:45:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.468+0000 7fb3bc200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b406e5e0 con 0x7fb3b4155df0 2024-08-17T02:45:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.468+0000 7fb3bc200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b40b0350 con 0x7fb3b4155a20 2024-08-17T02:45:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.468+0000 7fb3ba200640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb3b406df00 0x7fb3b4156b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46208/0 (socket says 172.21.15.16:46208) 2024-08-17T02:45:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.468+0000 7fb3ba200640 1 -- 172.21.15.16:0/1445992330 learned_addr learned my addr 172.21.15.16:0/1445992330 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.468+0000 7fb3b9800640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3b4155a20 0x7fb3b40adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.468+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647995331 0 0) 0x7fb3b4156670 con 0x7fb3b406df00 2024-08-17T02:45:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.468+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb39c0034e0 con 0x7fb3b406df00 2024-08-17T02:45:29.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.468+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64303188 0 0) 0x7fb3b406e5e0 con 0x7fb3b4155df0 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.468+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b4156670 con 0x7fb3b4155df0 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3941327163 0 0) 0x7fb39c0034e0 con 0x7fb3b406df00 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3b4155a20 msgr2=0x7fb3b40adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3b4155a20 0x7fb3b40adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 >> v1:172.21.15.142:6789/0 conn(0x7fb3b4155df0 legacy=0x7fb3b406d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b406e8c0 con 0x7fb3b406df00 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7fb3ac0017f0 con 0x7fb3b406df00 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 >> v1:172.21.15.16:6789/0 conn(0x7fb3b406df00 legacy=0x7fb3b4156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c003b90 0x7fb39c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb39c004780 0x7fb39c004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c005130 0x7fb39c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c00e1d0 con 0x7fb39c005130 2024-08-17T02:45:29.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c00e340 con 0x7fb39c004780 2024-08-17T02:45:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c00e500 con 0x7fb39c003b90 2024-08-17T02:45:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b9800640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb3b4155a20 0x7fb3b40adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3ba200640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c005130 0x7fb39c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b8e00640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb39c004780 0x7fb39c004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3b9800640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c003b90 0x7fb39c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3ba200640 1 -- 172.21.15.16:0/1445992330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c003b90 msgr2=0x7fb39c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3ba200640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c003b90 0x7fb39c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3ba200640 1 -- 172.21.15.16:0/1445992330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb39c004780 msgr2=0x7fb39c004bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3ba200640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb39c004780 0x7fb39c004bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.469+0000 7fb3ba200640 1 -- 172.21.15.16:0/1445992330 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c0034e0 con 0x7fb39c005130 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.470+0000 7fb3ba200640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c005130 0x7fb39c00dbe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb3b406e300 tx=0x7fb3ac0020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ef9f2af5b8622d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.471+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0071c0 con 0x7fb39c005130 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.471+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ac007340 con 0x7fb39c005130 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.471+0000 7fb3b3e00640 1 -- 172.21.15.16:0/1445992330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac007700 con 0x7fb39c005130 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.471+0000 7fb3bc200640 1 -- 172.21.15.16:0/1445992330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c005130 msgr2=0x7fb39c00dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.471+0000 7fb3bc200640 1 --2- 172.21.15.16:0/1445992330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb39c005130 0x7fb39c00dbe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb3b406e300 tx=0x7fb3ac0020c0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.471+0000 7fb3ba200640 1 -- 172.21.15.16:0/1445992330 reap_dead start 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.471+0000 7fb3bc200640 1 -- 172.21.15.16:0/1445992330 shutdown_connections 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.471+0000 7fb3bc200640 1 -- 172.21.15.16:0/1445992330 >> 172.21.15.16:0/1445992330 conn(0x7fb3b40cb9b0 msgr2=0x7fb3b40cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.471+0000 7fb3bc200640 1 -- 172.21.15.16:0/1445992330 shutdown_connections 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.471+0000 7fb3bc200640 1 -- 172.21.15.16:0/1445992330 wait complete. 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.473+0000 7fb3bc200640 1 Processor -- start 2024-08-17T02:45:29.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.473+0000 7fb3bc200640 1 -- start start 2024-08-17T02:45:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.474+0000 7fb3bc200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c005130 0x7fb3b413caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.474+0000 7fb3bc200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b40a98e0 0x7fb3b415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.474+0000 7fb3bc200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b415a700 0x7fb3b415ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.474+0000 7fb3bc200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b4168d20 con 0x7fb3b40a98e0 2024-08-17T02:45:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.474+0000 7fb3bc200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b41507d0 con 0x7fb3b415a700 2024-08-17T02:45:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.474+0000 7fb3bc200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b415cb60 con 0x7fb39c005130 2024-08-17T02:45:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.474+0000 7fb3b8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b40a98e0 0x7fb3b415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.474+0000 7fb3b8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b40a98e0 0x7fb3b415a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58310/0 (socket says 172.21.15.16:58310) 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.474+0000 7fb3b8e00640 1 -- 172.21.15.16:0/4058714396 learned_addr learned my addr 172.21.15.16:0/4058714396 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.474+0000 7fb3b9800640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c005130 0x7fb3b413caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.474+0000 7fb3ba200640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b415a700 0x7fb3b415ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.474+0000 7fb3b8e00640 1 -- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c005130 msgr2=0x7fb3b413caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.475+0000 7fb3b8e00640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c005130 0x7fb3b413caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.475+0000 7fb3b8e00640 1 -- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b415a700 msgr2=0x7fb3b415ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.475+0000 7fb3b8e00640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b415a700 0x7fb3b415ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.475+0000 7fb3b8e00640 1 -- 172.21.15.16:0/4058714396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b415ce20 con 0x7fb3b40a98e0 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.475+0000 7fb3b8e00640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b40a98e0 0x7fb3b415a1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0002bc0 tx=0x7fb3a0007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.475+0000 7fb3b0c00640 1 -- 172.21.15.16:0/4058714396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000fbb0 con 0x7fb3b40a98e0 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.475+0000 7fb3b0c00640 1 -- 172.21.15.16:0/4058714396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a000fd10 con 0x7fb3b40a98e0 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.475+0000 7fb3b0c00640 1 -- 172.21.15.16:0/4058714396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0007280 con 0x7fb3b40a98e0 2024-08-17T02:45:29.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.475+0000 7fb3bc200640 1 -- 172.21.15.16:0/4058714396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b415d100 con 0x7fb3b40a98e0 2024-08-17T02:45:29.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.475+0000 7fb3bc200640 1 -- 172.21.15.16:0/4058714396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b415d6c0 con 0x7fb3b40a98e0 2024-08-17T02:45:29.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.476+0000 7fb3bc200640 1 -- 172.21.15.16:0/4058714396 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b415d8e0 con 0x7fb3b40a98e0 2024-08-17T02:45:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.477+0000 7fb3b0c00640 1 -- 172.21.15.16:0/4058714396 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fb3a0006790 con 0x7fb3b40a98e0 2024-08-17T02:45:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.478+0000 7fb3b0c00640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb394065180 0x7fb394067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.478+0000 7fb3b0c00640 1 -- 172.21.15.16:0/4058714396 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb3a0082890 con 0x7fb3b40a98e0 2024-08-17T02:45:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.478+0000 7fb3b9800640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb394065180 0x7fb394067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.479+0000 7fb3b9800640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb394065180 0x7fb394067630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb3b406bc00 tx=0x7fb3a80090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:29.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.482+0000 7fb3b0c00640 1 -- 172.21.15.16:0/4058714396 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb3a00508b0 con 0x7fb3b40a98e0 2024-08-17T02:45:29.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.659+0000 7fb3bc200640 1 -- 172.21.15.16:0/4058714396 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3b4064db0 con 0x7fb394065180 2024-08-17T02:45:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.666+0000 7fb3b0c00640 1 -- 172.21.15.16:0/4058714396 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264562 (secure 0 0 0) 0x7fb3b4064db0 con 0x7fb394065180 2024-08-17T02:45:29.670 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-17T02:45:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.670+0000 7fb3bc200640 1 -- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb394065180 msgr2=0x7fb394067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.670+0000 7fb3bc200640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb394065180 0x7fb394067630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb3b406bc00 tx=0x7fb3a80090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.670+0000 7fb3bc200640 1 -- 172.21.15.16:0/4058714396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b40a98e0 msgr2=0x7fb3b415a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.670+0000 7fb3bc200640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b40a98e0 0x7fb3b415a1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0002bc0 tx=0x7fb3a0007c80 comp rx=0 tx=0).stop 2024-08-17T02:45:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.670+0000 7fb3bc200640 1 -- 172.21.15.16:0/4058714396 shutdown_connections 2024-08-17T02:45:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.670+0000 7fb3bc200640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb394065180 0x7fb394067630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.670+0000 7fb3bc200640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3b415a700 0x7fb3b415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.670+0000 7fb3bc200640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb3b40a98e0 0x7fb3b415a1c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.670+0000 7fb3bc200640 1 --2- 172.21.15.16:0/4058714396 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb39c005130 0x7fb3b413caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.670+0000 7fb3bc200640 1 -- 172.21.15.16:0/4058714396 >> 172.21.15.16:0/4058714396 conn(0x7fb3b40cb9b0 msgr2=0x7fb3b4154780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.671+0000 7fb3bc200640 1 -- 172.21.15.16:0/4058714396 shutdown_connections 2024-08-17T02:45:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.671+0000 7fb3bc200640 1 -- 172.21.15.16:0/4058714396 wait complete. 2024-08-17T02:45:29.686 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-08-17T02:45:28.237777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398428,"kb_used_data":2920,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752576292,"statfs":{"total":773094113280,"available":770638123008,"internally_reserved":0,"allocated":2990080,"data_stored":1787916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.224908"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048440+0000","last_change":"2024-08-17T02:45:20.044663+0000","last_active":"2024-08-17T02:45:20.048440+0000","last_peered":"2024-08-17T02:45:20.048440+0000","last_clean":"2024-08-17T02:45:20.048440+0000","last_became_active":"2024-08-17T02:45:20.044412+0000","last_became_peered":"2024-08-17T02:45:20.044412+0000","last_unstale":"2024-08-17T02:45:20.048440+0000","last_undegraded":"2024-08-17T02:45:20.048440+0000","last_fullsized":"2024-08-17T02:45:20.048440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:36:50.133729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.038551+0000","last_change":"2024-08-17T02:45:20.035833+0000","last_active":"2024-08-17T02:45:20.038551+0000","last_peered":"2024-08-17T02:45:20.038551+0000","last_clean":"2024-08-17T02:45:20.038551+0000","last_became_active":"2024-08-17T02:45:20.035557+0000","last_became_peered":"2024-08-17T02:45:20.035557+0000","last_unstale":"2024-08-17T02:45:20.038551+0000","last_undegraded":"2024-08-17T02:45:20.038551+0000","last_fullsized":"2024-08-17T02:45:20.038551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:16:04.128578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.031258+0000","last_change":"2024-08-17T02:45:20.027132+0000","last_active":"2024-08-17T02:45:20.031258+0000","last_peered":"2024-08-17T02:45:20.031258+0000","last_clean":"2024-08-17T02:45:20.031258+0000","last_became_active":"2024-08-17T02:45:20.026869+0000","last_became_peered":"2024-08-17T02:45:20.026869+0000","last_unstale":"2024-08-17T02:45:20.031258+0000","last_undegraded":"2024-08-17T02:45:20.031258+0000","last_fullsized":"2024-08-17T02:45:20.031258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:50:24.600922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.050217+0000","last_change":"2024-08-17T02:45:20.042821+0000","last_active":"2024-08-17T02:45:20.050217+0000","last_peered":"2024-08-17T02:45:20.050217+0000","last_clean":"2024-08-17T02:45:20.050217+0000","last_became_active":"2024-08-17T02:45:20.042556+0000","last_became_peered":"2024-08-17T02:45:20.042556+0000","last_unstale":"2024-08-17T02:45:20.050217+0000","last_undegraded":"2024-08-17T02:45:20.050217+0000","last_fullsized":"2024-08-17T02:45:20.050217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:04:44.206575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.034352+0000","last_change":"2024-08-17T02:45:20.030193+0000","last_active":"2024-08-17T02:45:20.034352+0000","last_peered":"2024-08-17T02:45:20.034352+0000","last_clean":"2024-08-17T02:45:20.034352+0000","last_became_active":"2024-08-17T02:45:20.029651+0000","last_became_peered":"2024-08-17T02:45:20.029651+0000","last_unstale":"2024-08-17T02:45:20.034352+0000","last_undegraded":"2024-08-17T02:45:20.034352+0000","last_fullsized":"2024-08-17T02:45:20.034352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:09:19.783891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047891+0000","last_change":"2024-08-17T02:45:20.040335+0000","last_active":"2024-08-17T02:45:20.047891+0000","last_peered":"2024-08-17T02:45:20.047891+0000","last_clean":"2024-08-17T02:45:20.047891+0000","last_became_active":"2024-08-17T02:45:20.039917+0000","last_became_peered":"2024-08-17T02:45:20.039917+0000","last_unstale":"2024-08-17T02:45:20.047891+0000","last_undegraded":"2024-08-17T02:45:20.047891+0000","last_fullsized":"2024-08-17T02:45:20.047891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:08.644038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047626+0000","last_change":"2024-08-17T02:45:20.036503+0000","last_active":"2024-08-17T02:45:20.047626+0000","last_peered":"2024-08-17T02:45:20.047626+0000","last_clean":"2024-08-17T02:45:20.047626+0000","last_became_active":"2024-08-17T02:45:20.033814+0000","last_became_peered":"2024-08-17T02:45:20.033814+0000","last_unstale":"2024-08-17T02:45:20.047626+0000","last_undegraded":"2024-08-17T02:45:20.047626+0000","last_fullsized":"2024-08-17T02:45:20.047626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:32:46.909589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051312+0000","last_change":"2024-08-17T02:45:20.046161+0000","last_active":"2024-08-17T02:45:20.051312+0000","last_peered":"2024-08-17T02:45:20.051312+0000","last_clean":"2024-08-17T02:45:20.051312+0000","last_became_active":"2024-08-17T02:45:20.042204+0000","last_became_peered":"2024-08-17T02:45:20.042204+0000","last_unstale":"2024-08-17T02:45:20.051312+0000","last_undegraded":"2024-08-17T02:45:20.051312+0000","last_fullsized":"2024-08-17T02:45:20.051312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:32:46.909589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.021896+0000","last_change":"2024-08-17T02:45:20.019114+0000","last_active":"2024-08-17T02:45:20.021896+0000","last_peered":"2024-08-17T02:45:20.021896+0000","last_clean":"2024-08-17T02:45:20.021896+0000","last_became_active":"2024-08-17T02:45:20.018864+0000","last_became_peered":"2024-08-17T02:45:20.018864+0000","last_unstale":"2024-08-17T02:45:20.021896+0000","last_undegraded":"2024-08-17T02:45:20.021896+0000","last_fullsized":"2024-08-17T02:45:20.021896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:05:58.485958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048643+0000","last_change":"2024-08-17T02:45:20.044934+0000","last_active":"2024-08-17T02:45:20.048643+0000","last_peered":"2024-08-17T02:45:20.048643+0000","last_clean":"2024-08-17T02:45:20.048643+0000","last_became_active":"2024-08-17T02:45:20.044408+0000","last_became_peered":"2024-08-17T02:45:20.044408+0000","last_unstale":"2024-08-17T02:45:20.048643+0000","last_undegraded":"2024-08-17T02:45:20.048643+0000","last_fullsized":"2024-08-17T02:45:20.048643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:50:24.600922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.049471+0000","last_change":"2024-08-17T02:45:20.044628+0000","last_active":"2024-08-17T02:45:20.049471+0000","last_peered":"2024-08-17T02:45:20.049471+0000","last_clean":"2024-08-17T02:45:20.049471+0000","last_became_active":"2024-08-17T02:45:20.044329+0000","last_became_peered":"2024-08-17T02:45:20.044329+0000","last_unstale":"2024-08-17T02:45:20.049471+0000","last_undegraded":"2024-08-17T02:45:20.049471+0000","last_fullsized":"2024-08-17T02:45:20.049471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:29:31.312177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048512+0000","last_change":"2024-08-17T02:45:20.042172+0000","last_active":"2024-08-17T02:45:20.048512+0000","last_peered":"2024-08-17T02:45:20.048512+0000","last_clean":"2024-08-17T02:45:20.048512+0000","last_became_active":"2024-08-17T02:45:20.041703+0000","last_became_peered":"2024-08-17T02:45:20.041703+0000","last_unstale":"2024-08-17T02:45:20.048512+0000","last_undegraded":"2024-08-17T02:45:20.048512+0000","last_fullsized":"2024-08-17T02:45:20.048512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:42.062952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052573+0000","last_change":"2024-08-17T02:45:20.045385+0000","last_active":"2024-08-17T02:45:20.052573+0000","last_peered":"2024-08-17T02:45:20.052573+0000","last_clean":"2024-08-17T02:45:20.052573+0000","last_became_active":"2024-08-17T02:45:20.045105+0000","last_became_peered":"2024-08-17T02:45:20.045105+0000","last_unstale":"2024-08-17T02:45:20.052573+0000","last_undegraded":"2024-08-17T02:45:20.052573+0000","last_fullsized":"2024-08-17T02:45:20.052573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:33:18.470465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051658+0000","last_change":"2024-08-17T02:45:20.039999+0000","last_active":"2024-08-17T02:45:20.051658+0000","last_peered":"2024-08-17T02:45:20.051658+0000","last_clean":"2024-08-17T02:45:20.051658+0000","last_became_active":"2024-08-17T02:45:20.039656+0000","last_became_peered":"2024-08-17T02:45:20.039656+0000","last_unstale":"2024-08-17T02:45:20.051658+0000","last_undegraded":"2024-08-17T02:45:20.051658+0000","last_fullsized":"2024-08-17T02:45:20.051658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:02:17.358927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.049636+0000","last_change":"2024-08-17T02:45:20.042360+0000","last_active":"2024-08-17T02:45:20.049636+0000","last_peered":"2024-08-17T02:45:20.049636+0000","last_clean":"2024-08-17T02:45:20.049636+0000","last_became_active":"2024-08-17T02:45:20.042078+0000","last_became_peered":"2024-08-17T02:45:20.042078+0000","last_unstale":"2024-08-17T02:45:20.049636+0000","last_undegraded":"2024-08-17T02:45:20.049636+0000","last_fullsized":"2024-08-17T02:45:20.049636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:41:49.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040453+0000","last_change":"2024-08-17T02:45:20.026340+0000","last_active":"2024-08-17T02:45:20.040453+0000","last_peered":"2024-08-17T02:45:20.040453+0000","last_clean":"2024-08-17T02:45:20.040453+0000","last_became_active":"2024-08-17T02:45:20.026045+0000","last_became_peered":"2024-08-17T02:45:20.026045+0000","last_unstale":"2024-08-17T02:45:20.040453+0000","last_undegraded":"2024-08-17T02:45:20.040453+0000","last_fullsized":"2024-08-17T02:45:20.040453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:15:44.912061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052119+0000","last_change":"2024-08-17T02:45:20.037915+0000","last_active":"2024-08-17T02:45:20.052119+0000","last_peered":"2024-08-17T02:45:20.052119+0000","last_clean":"2024-08-17T02:45:20.052119+0000","last_became_active":"2024-08-17T02:45:20.034495+0000","last_became_peered":"2024-08-17T02:45:20.034495+0000","last_unstale":"2024-08-17T02:45:20.052119+0000","last_undegraded":"2024-08-17T02:45:20.052119+0000","last_fullsized":"2024-08-17T02:45:20.052119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:37:11.925388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.056304+0000","last_change":"2024-08-17T02:45:20.054620+0000","last_active":"2024-08-17T02:45:20.056304+0000","last_peered":"2024-08-17T02:45:20.056304+0000","last_clean":"2024-08-17T02:45:20.056304+0000","last_became_active":"2024-08-17T02:45:20.054100+0000","last_became_peered":"2024-08-17T02:45:20.054100+0000","last_unstale":"2024-08-17T02:45:20.056304+0000","last_undegraded":"2024-08-17T02:45:20.056304+0000","last_fullsized":"2024-08-17T02:45:20.056304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:08.644038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.041022+0000","last_change":"2024-08-17T02:45:20.031250+0000","last_active":"2024-08-17T02:45:20.041022+0000","last_peered":"2024-08-17T02:45:20.041022+0000","last_clean":"2024-08-17T02:45:20.041022+0000","last_became_active":"2024-08-17T02:45:20.026261+0000","last_became_peered":"2024-08-17T02:45:20.026261+0000","last_unstale":"2024-08-17T02:45:20.041022+0000","last_undegraded":"2024-08-17T02:45:20.041022+0000","last_fullsized":"2024-08-17T02:45:20.041022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:54:59.143806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048503+0000","last_change":"2024-08-17T02:45:20.041999+0000","last_active":"2024-08-17T02:45:20.048503+0000","last_peered":"2024-08-17T02:45:20.048503+0000","last_clean":"2024-08-17T02:45:20.048503+0000","last_became_active":"2024-08-17T02:45:20.041438+0000","last_became_peered":"2024-08-17T02:45:20.041438+0000","last_unstale":"2024-08-17T02:45:20.048503+0000","last_undegraded":"2024-08-17T02:45:20.048503+0000","last_fullsized":"2024-08-17T02:45:20.048503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:20.121506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.032037+0000","last_change":"2024-08-17T02:45:20.027927+0000","last_active":"2024-08-17T02:45:20.032037+0000","last_peered":"2024-08-17T02:45:20.032037+0000","last_clean":"2024-08-17T02:45:20.032037+0000","last_became_active":"2024-08-17T02:45:20.026932+0000","last_became_peered":"2024-08-17T02:45:20.026932+0000","last_unstale":"2024-08-17T02:45:20.032037+0000","last_undegraded":"2024-08-17T02:45:20.032037+0000","last_fullsized":"2024-08-17T02:45:20.032037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:33:12.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.032654+0000","last_change":"2024-08-17T02:45:20.031118+0000","last_active":"2024-08-17T02:45:20.032654+0000","last_peered":"2024-08-17T02:45:20.032654+0000","last_clean":"2024-08-17T02:45:20.032654+0000","last_became_active":"2024-08-17T02:45:20.030216+0000","last_became_peered":"2024-08-17T02:45:20.030216+0000","last_unstale":"2024-08-17T02:45:20.032654+0000","last_undegraded":"2024-08-17T02:45:20.032654+0000","last_fullsized":"2024-08-17T02:45:20.032654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:41:33.774280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051361+0000","last_change":"2024-08-17T02:45:20.045808+0000","last_active":"2024-08-17T02:45:20.051361+0000","last_peered":"2024-08-17T02:45:20.051361+0000","last_clean":"2024-08-17T02:45:20.051361+0000","last_became_active":"2024-08-17T02:45:20.045029+0000","last_became_peered":"2024-08-17T02:45:20.045029+0000","last_unstale":"2024-08-17T02:45:20.051361+0000","last_undegraded":"2024-08-17T02:45:20.051361+0000","last_fullsized":"2024-08-17T02:45:20.051361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:30.359495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040472+0000","last_change":"2024-08-17T02:45:20.028434+0000","last_active":"2024-08-17T02:45:20.040472+0000","last_peered":"2024-08-17T02:45:20.040472+0000","last_clean":"2024-08-17T02:45:20.040472+0000","last_became_active":"2024-08-17T02:45:20.026769+0000","last_became_peered":"2024-08-17T02:45:20.026769+0000","last_unstale":"2024-08-17T02:45:20.040472+0000","last_undegraded":"2024-08-17T02:45:20.040472+0000","last_fullsized":"2024-08-17T02:45:20.040472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:40:57.163688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051347+0000","last_change":"2024-08-17T02:45:20.046140+0000","last_active":"2024-08-17T02:45:20.051347+0000","last_peered":"2024-08-17T02:45:20.051347+0000","last_clean":"2024-08-17T02:45:20.051347+0000","last_became_active":"2024-08-17T02:45:20.043780+0000","last_became_peered":"2024-08-17T02:45:20.043780+0000","last_unstale":"2024-08-17T02:45:20.051347+0000","last_undegraded":"2024-08-17T02:45:20.051347+0000","last_fullsized":"2024-08-17T02:45:20.051347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:15:44.912061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047741+0000","last_change":"2024-08-17T02:45:20.039484+0000","last_active":"2024-08-17T02:45:20.047741+0000","last_peered":"2024-08-17T02:45:20.047741+0000","last_clean":"2024-08-17T02:45:20.047741+0000","last_became_active":"2024-08-17T02:45:20.039202+0000","last_became_peered":"2024-08-17T02:45:20.039202+0000","last_unstale":"2024-08-17T02:45:20.047741+0000","last_undegraded":"2024-08-17T02:45:20.047741+0000","last_fullsized":"2024-08-17T02:45:20.047741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:37:57.900723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051555+0000","last_change":"2024-08-17T02:45:20.039636+0000","last_active":"2024-08-17T02:45:20.051555+0000","last_peered":"2024-08-17T02:45:20.051555+0000","last_clean":"2024-08-17T02:45:20.051555+0000","last_became_active":"2024-08-17T02:45:20.039357+0000","last_became_peered":"2024-08-17T02:45:20.039357+0000","last_unstale":"2024-08-17T02:45:20.051555+0000","last_undegraded":"2024-08-17T02:45:20.051555+0000","last_fullsized":"2024-08-17T02:45:20.051555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:53:40.660806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052416+0000","last_change":"2024-08-17T02:45:20.044576+0000","last_active":"2024-08-17T02:45:20.052416+0000","last_peered":"2024-08-17T02:45:20.052416+0000","last_clean":"2024-08-17T02:45:20.052416+0000","last_became_active":"2024-08-17T02:45:20.044338+0000","last_became_peered":"2024-08-17T02:45:20.044338+0000","last_unstale":"2024-08-17T02:45:20.052416+0000","last_undegraded":"2024-08-17T02:45:20.052416+0000","last_fullsized":"2024-08-17T02:45:20.052416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:33:12.091425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.049261+0000","last_change":"2024-08-17T02:45:20.041424+0000","last_active":"2024-08-17T02:45:20.049261+0000","last_peered":"2024-08-17T02:45:20.049261+0000","last_clean":"2024-08-17T02:45:20.049261+0000","last_became_active":"2024-08-17T02:45:20.041158+0000","last_became_peered":"2024-08-17T02:45:20.041158+0000","last_unstale":"2024-08-17T02:45:20.049261+0000","last_undegraded":"2024-08-17T02:45:20.049261+0000","last_fullsized":"2024-08-17T02:45:20.049261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:09:19.783891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.041640+0000","last_change":"2024-08-17T02:45:20.027101+0000","last_active":"2024-08-17T02:45:20.041640+0000","last_peered":"2024-08-17T02:45:20.041640+0000","last_clean":"2024-08-17T02:45:20.041640+0000","last_became_active":"2024-08-17T02:45:20.026792+0000","last_became_peered":"2024-08-17T02:45:20.026792+0000","last_unstale":"2024-08-17T02:45:20.041640+0000","last_undegraded":"2024-08-17T02:45:20.041640+0000","last_fullsized":"2024-08-17T02:45:20.041640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:27:01.489949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047767+0000","last_change":"2024-08-17T02:45:20.044023+0000","last_active":"2024-08-17T02:45:20.047767+0000","last_peered":"2024-08-17T02:45:20.047767+0000","last_clean":"2024-08-17T02:45:20.047767+0000","last_became_active":"2024-08-17T02:45:20.043731+0000","last_became_peered":"2024-08-17T02:45:20.043731+0000","last_unstale":"2024-08-17T02:45:20.047767+0000","last_undegraded":"2024-08-17T02:45:20.047767+0000","last_fullsized":"2024-08-17T02:45:20.047767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:41:33.774280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048891+0000","last_change":"2024-08-17T02:45:20.042822+0000","last_active":"2024-08-17T02:45:20.048891+0000","last_peered":"2024-08-17T02:45:20.048891+0000","last_clean":"2024-08-17T02:45:20.048891+0000","last_became_active":"2024-08-17T02:45:20.042501+0000","last_became_peered":"2024-08-17T02:45:20.042501+0000","last_unstale":"2024-08-17T02:45:20.048891+0000","last_undegraded":"2024-08-17T02:45:20.048891+0000","last_fullsized":"2024-08-17T02:45:20.048891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:53:40.660806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040933+0000","last_change":"2024-08-17T02:45:20.032549+0000","last_active":"2024-08-17T02:45:20.040933+0000","last_peered":"2024-08-17T02:45:20.040933+0000","last_clean":"2024-08-17T02:45:20.040933+0000","last_became_active":"2024-08-17T02:45:20.032196+0000","last_became_peered":"2024-08-17T02:45:20.032196+0000","last_unstale":"2024-08-17T02:45:20.040933+0000","last_undegraded":"2024-08-17T02:45:20.040933+0000","last_fullsized":"2024-08-17T02:45:20.040933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:08.644038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.053812+0000","last_change":"2024-08-17T02:45:19.040955+0000","last_active":"2024-08-17T02:45:20.053812+0000","last_peered":"2024-08-17T02:45:20.053812+0000","last_clean":"2024-08-17T02:45:20.053812+0000","last_became_active":"2024-08-17T02:45:19.039340+0000","last_became_peered":"2024-08-17T02:45:19.039340+0000","last_unstale":"2024-08-17T02:45:20.053812+0000","last_undegraded":"2024-08-17T02:45:20.053812+0000","last_fullsized":"2024-08-17T02:45:20.053812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040815+0000","last_change":"2024-08-17T02:45:20.035907+0000","last_active":"2024-08-17T02:45:20.040815+0000","last_peered":"2024-08-17T02:45:20.040815+0000","last_clean":"2024-08-17T02:45:20.040815+0000","last_became_active":"2024-08-17T02:45:20.035670+0000","last_became_peered":"2024-08-17T02:45:20.035670+0000","last_unstale":"2024-08-17T02:45:20.040815+0000","last_undegraded":"2024-08-17T02:45:20.040815+0000","last_fullsized":"2024-08-17T02:45:20.040815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:41:49.558998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054523+0000","last_change":"2024-08-17T02:45:19.048167+0000","last_active":"2024-08-17T02:45:20.054523+0000","last_peered":"2024-08-17T02:45:20.054523+0000","last_clean":"2024-08-17T02:45:20.054523+0000","last_became_active":"2024-08-17T02:45:19.047739+0000","last_became_peered":"2024-08-17T02:45:19.047739+0000","last_unstale":"2024-08-17T02:45:20.054523+0000","last_undegraded":"2024-08-17T02:45:20.054523+0000","last_fullsized":"2024-08-17T02:45:20.054523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:33:17.468626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.033343+0000","last_change":"2024-08-17T02:45:19.043960+0000","last_active":"2024-08-17T02:45:20.033343+0000","last_peered":"2024-08-17T02:45:20.033343+0000","last_clean":"2024-08-17T02:45:20.033343+0000","last_became_active":"2024-08-17T02:45:19.042684+0000","last_became_peered":"2024-08-17T02:45:19.042684+0000","last_unstale":"2024-08-17T02:45:20.033343+0000","last_undegraded":"2024-08-17T02:45:20.033343+0000","last_fullsized":"2024-08-17T02:45:20.033343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.038584+0000","last_change":"2024-08-17T02:45:20.034766+0000","last_active":"2024-08-17T02:45:20.038584+0000","last_peered":"2024-08-17T02:45:20.038584+0000","last_clean":"2024-08-17T02:45:20.038584+0000","last_became_active":"2024-08-17T02:45:20.034450+0000","last_became_peered":"2024-08-17T02:45:20.034450+0000","last_unstale":"2024-08-17T02:45:20.038584+0000","last_undegraded":"2024-08-17T02:45:20.038584+0000","last_fullsized":"2024-08-17T02:45:20.038584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:02:55.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.013694+0000","last_change":"2024-08-17T02:45:19.050133+0000","last_active":"2024-08-17T02:45:20.013694+0000","last_peered":"2024-08-17T02:45:20.013694+0000","last_clean":"2024-08-17T02:45:20.013694+0000","last_became_active":"2024-08-17T02:45:19.049873+0000","last_became_peered":"2024-08-17T02:45:19.049873+0000","last_unstale":"2024-08-17T02:45:20.013694+0000","last_undegraded":"2024-08-17T02:45:20.013694+0000","last_fullsized":"2024-08-17T02:45:20.013694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:05:57.484119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.031101+0000","last_change":"2024-08-17T02:45:20.023134+0000","last_active":"2024-08-17T02:45:20.031101+0000","last_peered":"2024-08-17T02:45:20.031101+0000","last_clean":"2024-08-17T02:45:20.031101+0000","last_became_active":"2024-08-17T02:45:20.022844+0000","last_became_peered":"2024-08-17T02:45:20.022844+0000","last_unstale":"2024-08-17T02:45:20.031101+0000","last_undegraded":"2024-08-17T02:45:20.031101+0000","last_fullsized":"2024-08-17T02:45:20.031101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:30.359495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.018045+0000","last_change":"2024-08-17T02:45:19.046922+0000","last_active":"2024-08-17T02:45:20.018045+0000","last_peered":"2024-08-17T02:45:20.018045+0000","last_clean":"2024-08-17T02:45:20.018045+0000","last_became_active":"2024-08-17T02:45:19.046578+0000","last_became_peered":"2024-08-17T02:45:19.046578+0000","last_unstale":"2024-08-17T02:45:20.018045+0000","last_undegraded":"2024-08-17T02:45:20.018045+0000","last_fullsized":"2024-08-17T02:45:20.018045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047730+0000","last_change":"2024-08-17T02:45:20.039463+0000","last_active":"2024-08-17T02:45:20.047730+0000","last_peered":"2024-08-17T02:45:20.047730+0000","last_clean":"2024-08-17T02:45:20.047730+0000","last_became_active":"2024-08-17T02:45:20.038869+0000","last_became_peered":"2024-08-17T02:45:20.038869+0000","last_unstale":"2024-08-17T02:45:20.047730+0000","last_undegraded":"2024-08-17T02:45:20.047730+0000","last_fullsized":"2024-08-17T02:45:20.047730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:33:18.470465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.035076+0000","last_change":"2024-08-17T02:45:19.048740+0000","last_active":"2024-08-17T02:45:20.035076+0000","last_peered":"2024-08-17T02:45:20.035076+0000","last_clean":"2024-08-17T02:45:20.035076+0000","last_became_active":"2024-08-17T02:45:19.047688+0000","last_became_peered":"2024-08-17T02:45:19.047688+0000","last_unstale":"2024-08-17T02:45:20.035076+0000","last_undegraded":"2024-08-17T02:45:20.035076+0000","last_fullsized":"2024-08-17T02:45:20.035076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:43.862231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051327+0000","last_change":"2024-08-17T02:45:20.046160+0000","last_active":"2024-08-17T02:45:20.051327+0000","last_peered":"2024-08-17T02:45:20.051327+0000","last_clean":"2024-08-17T02:45:20.051327+0000","last_became_active":"2024-08-17T02:45:20.045893+0000","last_became_peered":"2024-08-17T02:45:20.045893+0000","last_unstale":"2024-08-17T02:45:20.051327+0000","last_undegraded":"2024-08-17T02:45:20.051327+0000","last_fullsized":"2024-08-17T02:45:20.051327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:03:07.944484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054309+0000","last_change":"2024-08-17T02:45:19.043386+0000","last_active":"2024-08-17T02:45:20.054309+0000","last_peered":"2024-08-17T02:45:20.054309+0000","last_clean":"2024-08-17T02:45:20.054309+0000","last_became_active":"2024-08-17T02:45:19.043050+0000","last_became_peered":"2024-08-17T02:45:19.043050+0000","last_unstale":"2024-08-17T02:45:20.054309+0000","last_undegraded":"2024-08-17T02:45:20.054309+0000","last_fullsized":"2024-08-17T02:45:20.054309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.042043+0000","last_change":"2024-08-17T02:45:20.029004+0000","last_active":"2024-08-17T02:45:20.042043+0000","last_peered":"2024-08-17T02:45:20.042043+0000","last_clean":"2024-08-17T02:45:20.042043+0000","last_became_active":"2024-08-17T02:45:20.028698+0000","last_became_peered":"2024-08-17T02:45:20.028698+0000","last_unstale":"2024-08-17T02:45:20.042043+0000","last_undegraded":"2024-08-17T02:45:20.042043+0000","last_fullsized":"2024-08-17T02:45:20.042043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:42.062952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.058174+0000","last_change":"2024-08-17T02:45:19.034956+0000","last_active":"2024-08-17T02:45:20.058174+0000","last_peered":"2024-08-17T02:45:20.058174+0000","last_clean":"2024-08-17T02:45:20.058174+0000","last_became_active":"2024-08-17T02:45:19.034658+0000","last_became_peered":"2024-08-17T02:45:19.034658+0000","last_unstale":"2024-08-17T02:45:20.058174+0000","last_undegraded":"2024-08-17T02:45:20.058174+0000","last_fullsized":"2024-08-17T02:45:20.058174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048217+0000","last_change":"2024-08-17T02:45:20.039165+0000","last_active":"2024-08-17T02:45:20.048217+0000","last_peered":"2024-08-17T02:45:20.048217+0000","last_clean":"2024-08-17T02:45:20.048217+0000","last_became_active":"2024-08-17T02:45:20.038923+0000","last_became_peered":"2024-08-17T02:45:20.038923+0000","last_unstale":"2024-08-17T02:45:20.048217+0000","last_undegraded":"2024-08-17T02:45:20.048217+0000","last_fullsized":"2024-08-17T02:45:20.048217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:03:07.944484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.053604+0000","last_change":"2024-08-17T02:45:19.044788+0000","last_active":"2024-08-17T02:45:20.053604+0000","last_peered":"2024-08-17T02:45:20.053604+0000","last_clean":"2024-08-17T02:45:20.053604+0000","last_became_active":"2024-08-17T02:45:19.044538+0000","last_became_peered":"2024-08-17T02:45:19.044538+0000","last_unstale":"2024-08-17T02:45:20.053604+0000","last_undegraded":"2024-08-17T02:45:20.053604+0000","last_fullsized":"2024-08-17T02:45:20.053604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:44:52.129209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052358+0000","last_change":"2024-08-17T02:45:20.043190+0000","last_active":"2024-08-17T02:45:20.052358+0000","last_peered":"2024-08-17T02:45:20.052358+0000","last_clean":"2024-08-17T02:45:20.052358+0000","last_became_active":"2024-08-17T02:45:20.042860+0000","last_became_peered":"2024-08-17T02:45:20.042860+0000","last_unstale":"2024-08-17T02:45:20.052358+0000","last_undegraded":"2024-08-17T02:45:20.052358+0000","last_fullsized":"2024-08-17T02:45:20.052358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:55:18.896035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.013972+0000","last_change":"2024-08-17T02:45:19.047294+0000","last_active":"2024-08-17T02:45:20.013972+0000","last_peered":"2024-08-17T02:45:20.013972+0000","last_clean":"2024-08-17T02:45:20.013972+0000","last_became_active":"2024-08-17T02:45:19.042180+0000","last_became_peered":"2024-08-17T02:45:19.042180+0000","last_unstale":"2024-08-17T02:45:20.013972+0000","last_undegraded":"2024-08-17T02:45:20.013972+0000","last_fullsized":"2024-08-17T02:45:20.013972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:37:56.898883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052410+0000","last_change":"2024-08-17T02:45:20.038226+0000","last_active":"2024-08-17T02:45:20.052410+0000","last_peered":"2024-08-17T02:45:20.052410+0000","last_clean":"2024-08-17T02:45:20.052410+0000","last_became_active":"2024-08-17T02:45:20.034659+0000","last_became_peered":"2024-08-17T02:45:20.034659+0000","last_unstale":"2024-08-17T02:45:20.052410+0000","last_undegraded":"2024-08-17T02:45:20.052410+0000","last_fullsized":"2024-08-17T02:45:20.052410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:27:53.329893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.036680+0000","last_change":"2024-08-17T02:45:19.052673+0000","last_active":"2024-08-17T02:45:20.036680+0000","last_peered":"2024-08-17T02:45:20.036680+0000","last_clean":"2024-08-17T02:45:20.036680+0000","last_became_active":"2024-08-17T02:45:19.052404+0000","last_became_peered":"2024-08-17T02:45:19.052404+0000","last_unstale":"2024-08-17T02:45:20.036680+0000","last_undegraded":"2024-08-17T02:45:20.036680+0000","last_fullsized":"2024-08-17T02:45:20.036680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:46:09.850664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.039882+0000","last_change":"2024-08-17T02:45:20.031377+0000","last_active":"2024-08-17T02:45:20.039882+0000","last_peered":"2024-08-17T02:45:20.039882+0000","last_clean":"2024-08-17T02:45:20.039882+0000","last_became_active":"2024-08-17T02:45:20.031103+0000","last_became_peered":"2024-08-17T02:45:20.031103+0000","last_unstale":"2024-08-17T02:45:20.039882+0000","last_undegraded":"2024-08-17T02:45:20.039882+0000","last_fullsized":"2024-08-17T02:45:20.039882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:04:44.206575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054426+0000","last_change":"2024-08-17T02:45:19.042568+0000","last_active":"2024-08-17T02:45:20.054426+0000","last_peered":"2024-08-17T02:45:20.054426+0000","last_clean":"2024-08-17T02:45:20.054426+0000","last_became_active":"2024-08-17T02:45:19.041846+0000","last_became_peered":"2024-08-17T02:45:19.041846+0000","last_unstale":"2024-08-17T02:45:20.054426+0000","last_undegraded":"2024-08-17T02:45:20.054426+0000","last_fullsized":"2024-08-17T02:45:20.054426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.042968+0000","last_change":"2024-08-17T02:45:20.029516+0000","last_active":"2024-08-17T02:45:20.042968+0000","last_peered":"2024-08-17T02:45:20.042968+0000","last_clean":"2024-08-17T02:45:20.042968+0000","last_became_active":"2024-08-17T02:45:20.027167+0000","last_became_peered":"2024-08-17T02:45:20.027167+0000","last_unstale":"2024-08-17T02:45:20.042968+0000","last_undegraded":"2024-08-17T02:45:20.042968+0000","last_fullsized":"2024-08-17T02:45:20.042968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:20.121506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.056082+0000","last_change":"2024-08-17T02:45:19.038843+0000","last_active":"2024-08-17T02:45:20.056082+0000","last_peered":"2024-08-17T02:45:20.056082+0000","last_clean":"2024-08-17T02:45:20.056082+0000","last_became_active":"2024-08-17T02:45:19.038608+0000","last_became_peered":"2024-08-17T02:45:19.038608+0000","last_unstale":"2024-08-17T02:45:20.056082+0000","last_undegraded":"2024-08-17T02:45:20.056082+0000","last_fullsized":"2024-08-17T02:45:20.056082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:25:12.592566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047411+0000","last_change":"2024-08-17T02:45:20.037709+0000","last_active":"2024-08-17T02:45:20.047411+0000","last_peered":"2024-08-17T02:45:20.047411+0000","last_clean":"2024-08-17T02:45:20.047411+0000","last_became_active":"2024-08-17T02:45:20.037461+0000","last_became_peered":"2024-08-17T02:45:20.037461+0000","last_unstale":"2024-08-17T02:45:20.047411+0000","last_undegraded":"2024-08-17T02:45:20.047411+0000","last_fullsized":"2024-08-17T02:45:20.047411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:05:58.485958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051949+0000","last_change":"2024-08-17T02:45:19.039018+0000","last_active":"2024-08-17T02:45:20.051949+0000","last_peered":"2024-08-17T02:45:20.051949+0000","last_clean":"2024-08-17T02:45:20.051949+0000","last_became_active":"2024-08-17T02:45:19.036850+0000","last_became_peered":"2024-08-17T02:45:19.036850+0000","last_unstale":"2024-08-17T02:45:20.051949+0000","last_undegraded":"2024-08-17T02:45:20.051949+0000","last_fullsized":"2024-08-17T02:45:20.051949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.033842+0000","last_change":"2024-08-17T02:45:20.030655+0000","last_active":"2024-08-17T02:45:20.033842+0000","last_peered":"2024-08-17T02:45:20.033842+0000","last_clean":"2024-08-17T02:45:20.033842+0000","last_became_active":"2024-08-17T02:45:20.030439+0000","last_became_peered":"2024-08-17T02:45:20.030439+0000","last_unstale":"2024-08-17T02:45:20.033842+0000","last_undegraded":"2024-08-17T02:45:20.033842+0000","last_fullsized":"2024-08-17T02:45:20.033842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:15:44.912061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.055739+0000","last_change":"2024-08-17T02:45:19.045933+0000","last_active":"2024-08-17T02:45:20.055739+0000","last_peered":"2024-08-17T02:45:20.055739+0000","last_clean":"2024-08-17T02:45:20.055739+0000","last_became_active":"2024-08-17T02:45:19.045649+0000","last_became_peered":"2024-08-17T02:45:19.045649+0000","last_unstale":"2024-08-17T02:45:20.055739+0000","last_undegraded":"2024-08-17T02:45:20.055739+0000","last_fullsized":"2024-08-17T02:45:20.055739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.048520+0000","last_change":"2024-08-17T02:45:20.043745+0000","last_active":"2024-08-17T02:45:20.048520+0000","last_peered":"2024-08-17T02:45:20.048520+0000","last_clean":"2024-08-17T02:45:20.048520+0000","last_became_active":"2024-08-17T02:45:20.043470+0000","last_became_peered":"2024-08-17T02:45:20.043470+0000","last_unstale":"2024-08-17T02:45:20.048520+0000","last_undegraded":"2024-08-17T02:45:20.048520+0000","last_fullsized":"2024-08-17T02:45:20.048520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:00:06.622218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.033701+0000","last_change":"2024-08-17T02:45:19.040219+0000","last_active":"2024-08-17T02:45:20.033701+0000","last_peered":"2024-08-17T02:45:20.033701+0000","last_clean":"2024-08-17T02:45:20.033701+0000","last_became_active":"2024-08-17T02:45:19.039816+0000","last_became_peered":"2024-08-17T02:45:19.039816+0000","last_unstale":"2024-08-17T02:45:20.033701+0000","last_undegraded":"2024-08-17T02:45:20.033701+0000","last_fullsized":"2024-08-17T02:45:20.033701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.040997+0000","last_change":"2024-08-17T02:45:20.027155+0000","last_active":"2024-08-17T02:45:20.040997+0000","last_peered":"2024-08-17T02:45:20.040997+0000","last_clean":"2024-08-17T02:45:20.040997+0000","last_became_active":"2024-08-17T02:45:20.026102+0000","last_became_peered":"2024-08-17T02:45:20.026102+0000","last_unstale":"2024-08-17T02:45:20.040997+0000","last_undegraded":"2024-08-17T02:45:20.040997+0000","last_fullsized":"2024-08-17T02:45:20.040997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:05:05.162709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.017763+0000","last_change":"2024-08-17T02:45:16.001117+0000","last_active":"2024-08-17T02:45:20.017763+0000","last_peered":"2024-08-17T02:45:20.017763+0000","last_clean":"2024-08-17T02:45:20.017763+0000","last_became_active":"2024-08-17T02:45:16.000788+0000","last_became_peered":"2024-08-17T02:45:16.000788+0000","last_unstale":"2024-08-17T02:45:20.017763+0000","last_undegraded":"2024-08-17T02:45:20.017763+0000","last_fullsized":"2024-08-17T02:45:20.017763+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_clean_scrub_stamp":"2024-08-17T02:45:14.986707+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:27:37.295374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.014889+0000","last_change":"2024-08-17T02:45:19.042366+0000","last_active":"2024-08-17T02:45:20.014889+0000","last_peered":"2024-08-17T02:45:20.014889+0000","last_clean":"2024-08-17T02:45:20.014889+0000","last_became_active":"2024-08-17T02:45:19.039118+0000","last_became_peered":"2024-08-17T02:45:19.039118+0000","last_unstale":"2024-08-17T02:45:20.014889+0000","last_undegraded":"2024-08-17T02:45:20.014889+0000","last_fullsized":"2024-08-17T02:45:20.014889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:07.642199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.031922+0000","last_change":"2024-08-17T02:45:20.026830+0000","last_active":"2024-08-17T02:45:20.031922+0000","last_peered":"2024-08-17T02:45:20.031922+0000","last_clean":"2024-08-17T02:45:20.031922+0000","last_became_active":"2024-08-17T02:45:20.026135+0000","last_became_peered":"2024-08-17T02:45:20.026135+0000","last_unstale":"2024-08-17T02:45:20.031922+0000","last_undegraded":"2024-08-17T02:45:20.031922+0000","last_fullsized":"2024-08-17T02:45:20.031922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:37:57.900723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.013832+0000","last_change":"2024-08-17T02:45:19.046461+0000","last_active":"2024-08-17T02:45:20.013832+0000","last_peered":"2024-08-17T02:45:20.013832+0000","last_clean":"2024-08-17T02:45:20.013832+0000","last_became_active":"2024-08-17T02:45:19.042611+0000","last_became_peered":"2024-08-17T02:45:19.042611+0000","last_unstale":"2024-08-17T02:45:20.013832+0000","last_undegraded":"2024-08-17T02:45:20.013832+0000","last_fullsized":"2024-08-17T02:45:20.013832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:41:32.772441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052264+0000","last_change":"2024-08-17T02:45:20.043147+0000","last_active":"2024-08-17T02:45:20.052264+0000","last_peered":"2024-08-17T02:45:20.052264+0000","last_clean":"2024-08-17T02:45:20.052264+0000","last_became_active":"2024-08-17T02:45:20.042828+0000","last_became_peered":"2024-08-17T02:45:20.042828+0000","last_unstale":"2024-08-17T02:45:20.052264+0000","last_undegraded":"2024-08-17T02:45:20.052264+0000","last_fullsized":"2024-08-17T02:45:20.052264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:02:30.359495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.031941+0000","last_change":"2024-08-17T02:45:19.047594+0000","last_active":"2024-08-17T02:45:20.031941+0000","last_peered":"2024-08-17T02:45:20.031941+0000","last_clean":"2024-08-17T02:45:20.031941+0000","last_became_active":"2024-08-17T02:45:19.047271+0000","last_became_peered":"2024-08-17T02:45:19.047271+0000","last_unstale":"2024-08-17T02:45:20.031941+0000","last_undegraded":"2024-08-17T02:45:20.031941+0000","last_fullsized":"2024-08-17T02:45:20.031941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:59:04.410857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.050654+0000","last_change":"2024-08-17T02:45:20.043304+0000","last_active":"2024-08-17T02:45:20.050654+0000","last_peered":"2024-08-17T02:45:20.050654+0000","last_clean":"2024-08-17T02:45:20.050654+0000","last_became_active":"2024-08-17T02:45:20.038490+0000","last_became_peered":"2024-08-17T02:45:20.038490+0000","last_unstale":"2024-08-17T02:45:20.050654+0000","last_undegraded":"2024-08-17T02:45:20.050654+0000","last_fullsized":"2024-08-17T02:45:20.050654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:00:06.622218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.014144+0000","last_change":"2024-08-17T02:45:19.048638+0000","last_active":"2024-08-17T02:45:20.014144+0000","last_peered":"2024-08-17T02:45:20.014144+0000","last_clean":"2024-08-17T02:45:20.014144+0000","last_became_active":"2024-08-17T02:45:19.048346+0000","last_became_peered":"2024-08-17T02:45:19.048346+0000","last_unstale":"2024-08-17T02:45:20.014144+0000","last_undegraded":"2024-08-17T02:45:20.014144+0000","last_fullsized":"2024-08-17T02:45:20.014144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:46:09.850664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.034371+0000","last_change":"2024-08-17T02:45:20.030375+0000","last_active":"2024-08-17T02:45:20.034371+0000","last_peered":"2024-08-17T02:45:20.034371+0000","last_clean":"2024-08-17T02:45:20.034371+0000","last_became_active":"2024-08-17T02:45:20.029966+0000","last_became_peered":"2024-08-17T02:45:20.029966+0000","last_unstale":"2024-08-17T02:45:20.034371+0000","last_undegraded":"2024-08-17T02:45:20.034371+0000","last_fullsized":"2024-08-17T02:45:20.034371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:16:04.128578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.021330+0000","last_change":"2024-08-17T02:45:19.039495+0000","last_active":"2024-08-17T02:45:20.021330+0000","last_peered":"2024-08-17T02:45:20.021330+0000","last_clean":"2024-08-17T02:45:20.021330+0000","last_became_active":"2024-08-17T02:45:19.039218+0000","last_became_peered":"2024-08-17T02:45:19.039218+0000","last_unstale":"2024-08-17T02:45:20.021330+0000","last_undegraded":"2024-08-17T02:45:20.021330+0000","last_fullsized":"2024-08-17T02:45:20.021330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052935+0000","last_change":"2024-08-17T02:45:20.047183+0000","last_active":"2024-08-17T02:45:20.052935+0000","last_peered":"2024-08-17T02:45:20.052935+0000","last_clean":"2024-08-17T02:45:20.052935+0000","last_became_active":"2024-08-17T02:45:20.045627+0000","last_became_peered":"2024-08-17T02:45:20.045627+0000","last_unstale":"2024-08-17T02:45:20.052935+0000","last_undegraded":"2024-08-17T02:45:20.052935+0000","last_fullsized":"2024-08-17T02:45:20.052935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:02:17.358927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.016227+0000","last_change":"2024-08-17T02:45:19.043843+0000","last_active":"2024-08-17T02:45:20.016227+0000","last_peered":"2024-08-17T02:45:20.016227+0000","last_clean":"2024-08-17T02:45:20.016227+0000","last_became_active":"2024-08-17T02:45:19.043069+0000","last_became_peered":"2024-08-17T02:45:19.043069+0000","last_unstale":"2024-08-17T02:45:20.016227+0000","last_undegraded":"2024-08-17T02:45:20.016227+0000","last_fullsized":"2024-08-17T02:45:20.016227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:43.862231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051578+0000","last_change":"2024-08-17T02:45:20.038816+0000","last_active":"2024-08-17T02:45:20.051578+0000","last_peered":"2024-08-17T02:45:20.051578+0000","last_clean":"2024-08-17T02:45:20.051578+0000","last_became_active":"2024-08-17T02:45:20.038520+0000","last_became_peered":"2024-08-17T02:45:20.038520+0000","last_unstale":"2024-08-17T02:45:20.051578+0000","last_undegraded":"2024-08-17T02:45:20.051578+0000","last_fullsized":"2024-08-17T02:45:20.051578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:37:57.900723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-08-17T02:45:22.409624+0000","last_change":"2024-08-17T02:45:19.044921+0000","last_active":"2024-08-17T02:45:22.409624+0000","last_peered":"2024-08-17T02:45:22.409624+0000","last_clean":"2024-08-17T02:45:22.409624+0000","last_became_active":"2024-08-17T02:45:19.041305+0000","last_became_peered":"2024-08-17T02:45:19.041305+0000","last_unstale":"2024-08-17T02:45:22.409624+0000","last_undegraded":"2024-08-17T02:45:22.409624+0000","last_fullsized":"2024-08-17T02:45:22.409624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:03:27.020817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.041518+0000","last_change":"2024-08-17T02:45:20.035169+0000","last_active":"2024-08-17T02:45:20.041518+0000","last_peered":"2024-08-17T02:45:20.041518+0000","last_clean":"2024-08-17T02:45:20.041518+0000","last_became_active":"2024-08-17T02:45:20.034858+0000","last_became_peered":"2024-08-17T02:45:20.034858+0000","last_unstale":"2024-08-17T02:45:20.041518+0000","last_undegraded":"2024-08-17T02:45:20.041518+0000","last_fullsized":"2024-08-17T02:45:20.041518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:53:40.660806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054541+0000","last_change":"2024-08-17T02:45:19.049505+0000","last_active":"2024-08-17T02:45:20.054541+0000","last_peered":"2024-08-17T02:45:20.054541+0000","last_clean":"2024-08-17T02:45:20.054541+0000","last_became_active":"2024-08-17T02:45:19.048843+0000","last_became_peered":"2024-08-17T02:45:19.048843+0000","last_unstale":"2024-08-17T02:45:20.054541+0000","last_undegraded":"2024-08-17T02:45:20.054541+0000","last_fullsized":"2024-08-17T02:45:20.054541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:53:39.658967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.046988+0000","last_change":"2024-08-17T02:45:20.038861+0000","last_active":"2024-08-17T02:45:20.046988+0000","last_peered":"2024-08-17T02:45:20.046988+0000","last_clean":"2024-08-17T02:45:20.046988+0000","last_became_active":"2024-08-17T02:45:20.038597+0000","last_became_peered":"2024-08-17T02:45:20.038597+0000","last_unstale":"2024-08-17T02:45:20.046988+0000","last_undegraded":"2024-08-17T02:45:20.046988+0000","last_fullsized":"2024-08-17T02:45:20.046988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:02:55.963061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.034402+0000","last_change":"2024-08-17T02:45:19.035699+0000","last_active":"2024-08-17T02:45:20.034402+0000","last_peered":"2024-08-17T02:45:20.034402+0000","last_clean":"2024-08-17T02:45:20.034402+0000","last_became_active":"2024-08-17T02:45:19.035429+0000","last_became_peered":"2024-08-17T02:45:19.035429+0000","last_unstale":"2024-08-17T02:45:20.034402+0000","last_undegraded":"2024-08-17T02:45:20.034402+0000","last_fullsized":"2024-08-17T02:45:20.034402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:41:32.772441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.021682+0000","last_change":"2024-08-17T02:45:20.020728+0000","last_active":"2024-08-17T02:45:20.021682+0000","last_peered":"2024-08-17T02:45:20.021682+0000","last_clean":"2024-08-17T02:45:20.021682+0000","last_became_active":"2024-08-17T02:45:20.020439+0000","last_became_peered":"2024-08-17T02:45:20.020439+0000","last_unstale":"2024-08-17T02:45:20.021682+0000","last_undegraded":"2024-08-17T02:45:20.021682+0000","last_fullsized":"2024-08-17T02:45:20.021682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:32:46.909589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.016454+0000","last_change":"2024-08-17T02:45:19.044294+0000","last_active":"2024-08-17T02:45:20.016454+0000","last_peered":"2024-08-17T02:45:20.016454+0000","last_clean":"2024-08-17T02:45:20.016454+0000","last_became_active":"2024-08-17T02:45:19.044001+0000","last_became_peered":"2024-08-17T02:45:19.044001+0000","last_unstale":"2024-08-17T02:45:20.016454+0000","last_undegraded":"2024-08-17T02:45:20.016454+0000","last_fullsized":"2024-08-17T02:45:20.016454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:56:43.862231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.047889+0000","last_change":"2024-08-17T02:45:20.040445+0000","last_active":"2024-08-17T02:45:20.047889+0000","last_peered":"2024-08-17T02:45:20.047889+0000","last_clean":"2024-08-17T02:45:20.047889+0000","last_became_active":"2024-08-17T02:45:20.040151+0000","last_became_peered":"2024-08-17T02:45:20.040151+0000","last_unstale":"2024-08-17T02:45:20.047889+0000","last_undegraded":"2024-08-17T02:45:20.047889+0000","last_fullsized":"2024-08-17T02:45:20.047889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:55:18.896035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.033393+0000","last_change":"2024-08-17T02:45:19.037364+0000","last_active":"2024-08-17T02:45:20.033393+0000","last_peered":"2024-08-17T02:45:20.033393+0000","last_clean":"2024-08-17T02:45:20.033393+0000","last_became_active":"2024-08-17T02:45:19.037020+0000","last_became_peered":"2024-08-17T02:45:19.037020+0000","last_unstale":"2024-08-17T02:45:20.033393+0000","last_undegraded":"2024-08-17T02:45:20.033393+0000","last_fullsized":"2024-08-17T02:45:20.033393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:25:48.022943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052107+0000","last_change":"2024-08-17T02:45:20.047671+0000","last_active":"2024-08-17T02:45:20.052107+0000","last_peered":"2024-08-17T02:45:20.052107+0000","last_clean":"2024-08-17T02:45:20.052107+0000","last_became_active":"2024-08-17T02:45:20.046596+0000","last_became_peered":"2024-08-17T02:45:20.046596+0000","last_unstale":"2024-08-17T02:45:20.052107+0000","last_undegraded":"2024-08-17T02:45:20.052107+0000","last_fullsized":"2024-08-17T02:45:20.052107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:46:10.852503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.057610+0000","last_change":"2024-08-17T02:45:19.040794+0000","last_active":"2024-08-17T02:45:20.057610+0000","last_peered":"2024-08-17T02:45:20.057610+0000","last_clean":"2024-08-17T02:45:20.057610+0000","last_became_active":"2024-08-17T02:45:19.038764+0000","last_became_peered":"2024-08-17T02:45:19.038764+0000","last_unstale":"2024-08-17T02:45:20.057610+0000","last_undegraded":"2024-08-17T02:45:20.057610+0000","last_fullsized":"2024-08-17T02:45:20.057610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:04:43.204736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.021022+0000","last_change":"2024-08-17T02:45:20.019301+0000","last_active":"2024-08-17T02:45:20.021022+0000","last_peered":"2024-08-17T02:45:20.021022+0000","last_clean":"2024-08-17T02:45:20.021022+0000","last_became_active":"2024-08-17T02:45:20.018641+0000","last_became_peered":"2024-08-17T02:45:20.018641+0000","last_unstale":"2024-08-17T02:45:20.021022+0000","last_undegraded":"2024-08-17T02:45:20.021022+0000","last_fullsized":"2024-08-17T02:45:20.021022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:47:06.908123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.052207+0000","last_change":"2024-08-17T02:45:20.037807+0000","last_active":"2024-08-17T02:45:20.052207+0000","last_peered":"2024-08-17T02:45:20.052207+0000","last_clean":"2024-08-17T02:45:20.052207+0000","last_became_active":"2024-08-17T02:45:20.034485+0000","last_became_peered":"2024-08-17T02:45:20.034485+0000","last_unstale":"2024-08-17T02:45:20.052207+0000","last_undegraded":"2024-08-17T02:45:20.052207+0000","last_fullsized":"2024-08-17T02:45:20.052207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:29:54.794188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.055868+0000","last_change":"2024-08-17T02:45:19.046881+0000","last_active":"2024-08-17T02:45:20.055868+0000","last_peered":"2024-08-17T02:45:20.055868+0000","last_clean":"2024-08-17T02:45:20.055868+0000","last_became_active":"2024-08-17T02:45:19.046676+0000","last_became_peered":"2024-08-17T02:45:19.046676+0000","last_unstale":"2024-08-17T02:45:20.055868+0000","last_undegraded":"2024-08-17T02:45:20.055868+0000","last_fullsized":"2024-08-17T02:45:20.055868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:29:42.855891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.041726+0000","last_change":"2024-08-17T02:45:20.040519+0000","last_active":"2024-08-17T02:45:20.041726+0000","last_peered":"2024-08-17T02:45:20.041726+0000","last_clean":"2024-08-17T02:45:20.041726+0000","last_became_active":"2024-08-17T02:45:20.040246+0000","last_became_peered":"2024-08-17T02:45:20.040246+0000","last_unstale":"2024-08-17T02:45:20.041726+0000","last_undegraded":"2024-08-17T02:45:20.041726+0000","last_fullsized":"2024-08-17T02:45:20.041726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:46:10.852503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.055789+0000","last_change":"2024-08-17T02:45:19.045919+0000","last_active":"2024-08-17T02:45:20.055789+0000","last_peered":"2024-08-17T02:45:20.055789+0000","last_clean":"2024-08-17T02:45:20.055789+0000","last_became_active":"2024-08-17T02:45:19.045693+0000","last_became_peered":"2024-08-17T02:45:19.045693+0000","last_unstale":"2024-08-17T02:45:20.055789+0000","last_undegraded":"2024-08-17T02:45:20.055789+0000","last_fullsized":"2024-08-17T02:45:20.055789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:55:24.029624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.050681+0000","last_change":"2024-08-17T02:45:20.044015+0000","last_active":"2024-08-17T02:45:20.050681+0000","last_peered":"2024-08-17T02:45:20.050681+0000","last_clean":"2024-08-17T02:45:20.050681+0000","last_became_active":"2024-08-17T02:45:20.043471+0000","last_became_peered":"2024-08-17T02:45:20.043471+0000","last_unstale":"2024-08-17T02:45:20.050681+0000","last_undegraded":"2024-08-17T02:45:20.050681+0000","last_fullsized":"2024-08-17T02:45:20.050681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:56:20.121506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054182+0000","last_change":"2024-08-17T02:45:19.041823+0000","last_active":"2024-08-17T02:45:20.054182+0000","last_peered":"2024-08-17T02:45:20.054182+0000","last_clean":"2024-08-17T02:45:20.054182+0000","last_became_active":"2024-08-17T02:45:19.041481+0000","last_became_peered":"2024-08-17T02:45:19.041481+0000","last_unstale":"2024-08-17T02:45:20.054182+0000","last_undegraded":"2024-08-17T02:45:20.054182+0000","last_fullsized":"2024-08-17T02:45:20.054182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:03:27.020817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.051425+0000","last_change":"2024-08-17T02:45:20.047339+0000","last_active":"2024-08-17T02:45:20.051425+0000","last_peered":"2024-08-17T02:45:20.051425+0000","last_clean":"2024-08-17T02:45:20.051425+0000","last_became_active":"2024-08-17T02:45:20.047111+0000","last_became_peered":"2024-08-17T02:45:20.047111+0000","last_unstale":"2024-08-17T02:45:20.051425+0000","last_undegraded":"2024-08-17T02:45:20.051425+0000","last_fullsized":"2024-08-17T02:45:20.051425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:19.012559+0000","last_clean_scrub_stamp":"2024-08-17T02:45:19.012559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:46:59.247481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-08-17T02:45:20.054521+0000","last_change":"2024-08-17T02:45:19.042331+0000","last_active":"2024-08-17T02:45:20.054521+0000","last_peered":"2024-08-17T02:45:20.054521+0000","last_clean":"2024-08-17T02:45:20.054521+0000","last_became_active":"2024-08-17T02:45:19.042033+0000","last_became_peered":"2024-08-17T02:45:19.042033+0000","last_unstale":"2024-08-17T02:45:20.054521+0000","last_undegraded":"2024-08-17T02:45:20.054521+0000","last_fullsized":"2024-08-17T02:45:20.054521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:18.010720+0000","last_clean_scrub_stamp":"2024-08-17T02:45:18.010720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:25:12.592566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":75437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297568,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074272,"statfs":{"total":96636764160,"available":96332054528,"internally_reserved":0,"allocated":217088,"data_stored":75313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298156,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073684,"statfs":{"total":96636764160,"available":96331452416,"internally_reserved":0,"allocated":819200,"data_stored":665773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":237568,"data_stored":77525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":221184,"data_stored":75883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301744,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070096,"statfs":{"total":96636764160,"available":96327778304,"internally_reserved":0,"allocated":229376,"data_stored":76865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":815104,"data_stored":665751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0449999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301740,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070100,"statfs":{"total":96636764160,"available":96327782400,"internally_reserved":0,"allocated":225280,"data_stored":75369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:45:29.690 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-17T02:45:29.691 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-17T02:45:29.691 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-08-17T02:45:29.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-08-17T02:45:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.876+0000 7fa72fe00640 1 Processor -- start 2024-08-17T02:45:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.876+0000 7fa72fe00640 1 -- start start 2024-08-17T02:45:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.877+0000 7fa72fe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa728158f80 0x7fa7280ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.877+0000 7fa72fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7280cf280 0x7fa7280ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.877+0000 7fa72fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7280cb560 con 0x7fa728159350 2024-08-17T02:45:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.877+0000 7fa72fe00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7280cf7e0 con 0x7fa728158f80 2024-08-17T02:45:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.877+0000 7fa72fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7280cfac0 con 0x7fa7280cf280 2024-08-17T02:45:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.877+0000 7fa72d400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa728158f80 0x7fa7280ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.877+0000 7fa72d400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa728158f80 0x7fa7280ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58320/0 (socket says 172.21.15.16:58320) 2024-08-17T02:45:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.877+0000 7fa72d400640 1 -- 172.21.15.16:0/1825532355 learned_addr learned my addr 172.21.15.16:0/1825532355 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.877+0000 7fa72de00640 1 --2- 172.21.15.16:0/1825532355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7280cf280 0x7fa7280ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.878+0000 7fa72d400640 1 -- 172.21.15.16:0/1825532355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7280cf280 msgr2=0x7fa7280ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.878+0000 7fa72d400640 1 --2- 172.21.15.16:0/1825532355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7280cf280 0x7fa7280ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.878+0000 7fa72d400640 1 -- 172.21.15.16:0/1825532355 >> v1:172.21.15.142:6789/0 conn(0x7fa728159350 legacy=0x7fa7280cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.878+0000 7fa72d400640 1 -- 172.21.15.16:0/1825532355 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7280caf10 con 0x7fa728158f80 2024-08-17T02:45:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.879+0000 7fa72d400640 1 --2- 172.21.15.16:0/1825532355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa728158f80 0x7fa7280ca330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009910 tx=0x7fa71c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e662b9f71dd9837 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.879+0000 7fa727e00640 1 -- 172.21.15.16:0/1825532355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c02ae40 con 0x7fa728158f80 2024-08-17T02:45:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.879+0000 7fa727e00640 1 -- 172.21.15.16:0/1825532355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c032440 con 0x7fa728158f80 2024-08-17T02:45:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.879+0000 7fa727e00640 1 -- 172.21.15.16:0/1825532355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c032820 con 0x7fa728158f80 2024-08-17T02:45:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.880+0000 7fa72fe00640 1 -- 172.21.15.16:0/1825532355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa728158f80 msgr2=0x7fa7280ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.880+0000 7fa72fe00640 1 --2- 172.21.15.16:0/1825532355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa728158f80 0x7fa7280ca330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009910 tx=0x7fa71c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.880+0000 7fa72fe00640 1 -- 172.21.15.16:0/1825532355 shutdown_connections 2024-08-17T02:45:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.880+0000 7fa72fe00640 1 --2- 172.21.15.16:0/1825532355 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7280cf280 0x7fa7280ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.880+0000 7fa72fe00640 1 --2- 172.21.15.16:0/1825532355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa728158f80 0x7fa7280ca330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.880+0000 7fa72fe00640 1 -- 172.21.15.16:0/1825532355 >> 172.21.15.16:0/1825532355 conn(0x7fa7280cfd90 msgr2=0x7fa7280d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.880+0000 7fa72fe00640 1 -- 172.21.15.16:0/1825532355 shutdown_connections 2024-08-17T02:45:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.880+0000 7fa72fe00640 1 -- 172.21.15.16:0/1825532355 wait complete. 2024-08-17T02:45:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.881+0000 7fa72fe00640 1 Processor -- start 2024-08-17T02:45:29.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.881+0000 7fa72fe00640 1 -- start start 2024-08-17T02:45:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.882+0000 7fa72fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7280cf280 0x7fa728146150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.882+0000 7fa72fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa728158f80 0x7fa7280c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.882+0000 7fa72fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa728159350 0x7fa728067cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.882+0000 7fa72fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa728146690 con 0x7fa728158f80 2024-08-17T02:45:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.882+0000 7fa72fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7280c59c0 con 0x7fa728159350 2024-08-17T02:45:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.882+0000 7fa72fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7280c5ca0 con 0x7fa7280cf280 2024-08-17T02:45:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.882+0000 7fa72ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa728158f80 0x7fa7280c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.882+0000 7fa72ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa728158f80 0x7fa7280c4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58322/0 (socket says 172.21.15.16:58322) 2024-08-17T02:45:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.882+0000 7fa72ca00640 1 -- 172.21.15.16:0/3219520727 learned_addr learned my addr 172.21.15.16:0/3219520727 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.882+0000 7fa72d400640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7280cf280 0x7fa728146150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.882+0000 7fa72de00640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa728159350 0x7fa728067cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.883+0000 7fa72ca00640 1 -- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7280cf280 msgr2=0x7fa728146150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.883+0000 7fa72ca00640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7280cf280 0x7fa728146150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.883+0000 7fa72ca00640 1 -- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa728159350 msgr2=0x7fa728067cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.883+0000 7fa72ca00640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa728159350 0x7fa728067cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:29.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.883+0000 7fa72ca00640 1 -- 172.21.15.16:0/3219520727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c009450 con 0x7fa728158f80 2024-08-17T02:45:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.883+0000 7fa72ca00640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa728158f80 0x7fa7280c4f70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c560 tx=0x7fa714007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.883+0000 7fa724c00640 1 -- 172.21.15.16:0/3219520727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714001120 con 0x7fa728158f80 2024-08-17T02:45:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.883+0000 7fa724c00640 1 -- 172.21.15.16:0/3219520727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa714011590 con 0x7fa728158f80 2024-08-17T02:45:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.883+0000 7fa724c00640 1 -- 172.21.15.16:0/3219520727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714011950 con 0x7fa728158f80 2024-08-17T02:45:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.883+0000 7fa72fe00640 1 -- 172.21.15.16:0/3219520727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7280c5f80 con 0x7fa728158f80 2024-08-17T02:45:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.884+0000 7fa72fe00640 1 -- 172.21.15.16:0/3219520727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7280c6540 con 0x7fa728158f80 2024-08-17T02:45:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.885+0000 7fa72fe00640 1 -- 172.21.15.16:0/3219520727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa728159750 con 0x7fa728158f80 2024-08-17T02:45:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.885+0000 7fa724c00640 1 -- 172.21.15.16:0/3219520727 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fa71401b3b0 con 0x7fa728158f80 2024-08-17T02:45:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.886+0000 7fa724c00640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa708065180 0x7fa708067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:29.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.886+0000 7fa724c00640 1 -- 172.21.15.16:0/3219520727 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa714081920 con 0x7fa728158f80 2024-08-17T02:45:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.887+0000 7fa72d400640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa708065180 0x7fa708067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.887+0000 7fa72d400640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa708065180 0x7fa708067630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa71c034000 tx=0x7fa71c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:29.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:29.890+0000 7fa724c00640 1 -- 172.21.15.16:0/3219520727 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa71404f9f0 con 0x7fa728158f80 2024-08-17T02:45:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.107+0000 7fa72fe00640 1 -- 172.21.15.16:0/3219520727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa728064db0 con 0x7fa728158f80 2024-08-17T02:45:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.107+0000 7fa724c00640 1 -- 172.21.15.16:0/3219520727 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa714017020 con 0x7fa728158f80 2024-08-17T02:45:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:30.109 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-17T02:45:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.109+0000 7fa72fe00640 1 -- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa708065180 msgr2=0x7fa708067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.109+0000 7fa72fe00640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa708065180 0x7fa708067630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa71c034000 tx=0x7fa71c032000 comp rx=0 tx=0).stop 2024-08-17T02:45:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.109+0000 7fa72fe00640 1 -- 172.21.15.16:0/3219520727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa728158f80 msgr2=0x7fa7280c4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.109+0000 7fa72fe00640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa728158f80 0x7fa7280c4f70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c560 tx=0x7fa714007ef0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.109+0000 7fa72fe00640 1 -- 172.21.15.16:0/3219520727 shutdown_connections 2024-08-17T02:45:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.109+0000 7fa72fe00640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa708065180 0x7fa708067630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.109+0000 7fa72fe00640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa728159350 0x7fa728067cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.109+0000 7fa72fe00640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa728158f80 0x7fa7280c4f70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.109+0000 7fa72fe00640 1 --2- 172.21.15.16:0/3219520727 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa7280cf280 0x7fa728146150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.109+0000 7fa72fe00640 1 -- 172.21.15.16:0/3219520727 >> 172.21.15.16:0/3219520727 conn(0x7fa7280cfd90 msgr2=0x7fa7280adc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.109+0000 7fa72fe00640 1 -- 172.21.15.16:0/3219520727 shutdown_connections 2024-08-17T02:45:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.109+0000 7fa72fe00640 1 -- 172.21.15.16:0/3219520727 wait complete. 2024-08-17T02:45:30.125 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-08-17T02:45:30.125 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:30.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.274+0000 7f7abe200640 1 Processor -- start 2024-08-17T02:45:30.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.274+0000 7f7abe200640 1 -- start start 2024-08-17T02:45:30.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.275+0000 7f7abe200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7ab81401b0 0x7f7ab80caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:30.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.275+0000 7f7abe200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ab8140580 0x7f7ab80b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:30.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.275+0000 7f7abe200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab80b16d0 con 0x7f7ab80b1270 2024-08-17T02:45:30.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.275+0000 7f7abe200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab80cb450 con 0x7f7ab8140580 2024-08-17T02:45:30.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.275+0000 7f7abe200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab80caa80 con 0x7f7ab81401b0 2024-08-17T02:45:30.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.275+0000 7f7ab6a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ab8140580 0x7f7ab80b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:30.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.275+0000 7f7ab6a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ab8140580 0x7f7ab80b0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58324/0 (socket says 172.21.15.16:58324) 2024-08-17T02:45:30.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.275+0000 7f7ab6a00640 1 -- 172.21.15.16:0/575752171 learned_addr learned my addr 172.21.15.16:0/575752171 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:30.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.275+0000 7f7ab7400640 1 --2- 172.21.15.16:0/575752171 >> v2:172.21.15.142:3300/0 conn(0x7f7ab81401b0 0x7f7ab80caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:30.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.276+0000 7f7ab6a00640 1 -- 172.21.15.16:0/575752171 >> v2:172.21.15.142:3300/0 conn(0x7f7ab81401b0 msgr2=0x7f7ab80caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:30.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.276+0000 7f7ab6a00640 1 --2- 172.21.15.16:0/575752171 >> v2:172.21.15.142:3300/0 conn(0x7f7ab81401b0 0x7f7ab80caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.276+0000 7f7ab6a00640 1 -- 172.21.15.16:0/575752171 >> v1:172.21.15.142:6789/0 conn(0x7f7ab80b1270 legacy=0x7f7ab80c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.276+0000 7f7ab6a00640 1 -- 172.21.15.16:0/575752171 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab80c9e70 con 0x7f7ab8140580 2024-08-17T02:45:30.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.277+0000 7f7ab6a00640 1 --2- 172.21.15.16:0/575752171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab8140580 0x7f7ab80b0d30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8068650 tx=0x7f7aa40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a03d9f69ddcad6 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:30.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.277+0000 7f7ab6000640 1 -- 172.21.15.16:0/575752171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa402adf0 con 0x7f7ab8140580 2024-08-17T02:45:30.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.277+0000 7f7ab6000640 1 -- 172.21.15.16:0/575752171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa4032440 con 0x7f7ab8140580 2024-08-17T02:45:30.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.277+0000 7f7ab6000640 1 -- 172.21.15.16:0/575752171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4032820 con 0x7f7ab8140580 2024-08-17T02:45:30.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.278+0000 7f7abe200640 1 -- 172.21.15.16:0/575752171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab8140580 msgr2=0x7f7ab80b0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.278+0000 7f7abe200640 1 --2- 172.21.15.16:0/575752171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab8140580 0x7f7ab80b0d30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8068650 tx=0x7f7aa40091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.278+0000 7f7abe200640 1 -- 172.21.15.16:0/575752171 shutdown_connections 2024-08-17T02:45:30.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.278+0000 7f7abe200640 1 --2- 172.21.15.16:0/575752171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab8140580 0x7f7ab80b0d30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.278+0000 7f7abe200640 1 --2- 172.21.15.16:0/575752171 >> v2:172.21.15.142:3300/0 conn(0x7f7ab81401b0 0x7f7ab80caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.278+0000 7f7abe200640 1 -- 172.21.15.16:0/575752171 >> 172.21.15.16:0/575752171 conn(0x7f7ab80cb9b0 msgr2=0x7f7ab80cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:30.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.278+0000 7f7abe200640 1 -- 172.21.15.16:0/575752171 shutdown_connections 2024-08-17T02:45:30.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.278+0000 7f7abe200640 1 -- 172.21.15.16:0/575752171 wait complete. 2024-08-17T02:45:30.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.279+0000 7f7abe200640 1 Processor -- start 2024-08-17T02:45:30.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.279+0000 7f7abe200640 1 -- start start 2024-08-17T02:45:30.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.280+0000 7f7abe200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab80b1270 0x7f7ab813ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.280+0000 7f7abe200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ab81401b0 0x7f7ab813f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.280+0000 7f7abe200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ab8140580 0x7f7ab8069d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.280+0000 7f7abe200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab80af280 con 0x7f7ab80b1270 2024-08-17T02:45:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.280+0000 7f7abe200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab80aecc0 con 0x7f7ab8140580 2024-08-17T02:45:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.280+0000 7f7abe200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab80aefa0 con 0x7f7ab81401b0 2024-08-17T02:45:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.280+0000 7f7ab7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab80b1270 0x7f7ab813ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.280+0000 7f7ab7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab80b1270 0x7f7ab813ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58328/0 (socket says 172.21.15.16:58328) 2024-08-17T02:45:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.280+0000 7f7ab7400640 1 -- 172.21.15.16:0/4021156616 learned_addr learned my addr 172.21.15.16:0/4021156616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.280+0000 7f7ab6a00640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ab81401b0 0x7f7ab813f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.281+0000 7f7ab7e00640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ab8140580 0x7f7ab8069d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.281+0000 7f7ab7400640 1 -- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ab81401b0 msgr2=0x7f7ab813f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.281+0000 7f7ab7400640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ab81401b0 0x7f7ab813f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.281+0000 7f7ab7400640 1 -- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ab8140580 msgr2=0x7f7ab8069d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.281+0000 7f7ab7400640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ab8140580 0x7f7ab8069d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.281+0000 7f7ab7400640 1 -- 172.21.15.16:0/4021156616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa4009450 con 0x7f7ab80b1270 2024-08-17T02:45:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.281+0000 7f7ab7e00640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ab8140580 0x7f7ab8069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.281+0000 7f7ab7400640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab80b1270 0x7f7ab813ef50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800ea20 tx=0x7f7aa80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.281+0000 7f7aaea00640 1 -- 172.21.15.16:0/4021156616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800ca10 con 0x7f7ab80b1270 2024-08-17T02:45:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.281+0000 7f7aaea00640 1 -- 172.21.15.16:0/4021156616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa800cb70 con 0x7f7ab80b1270 2024-08-17T02:45:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.281+0000 7f7aaea00640 1 -- 172.21.15.16:0/4021156616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8013470 con 0x7f7ab80b1270 2024-08-17T02:45:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.281+0000 7f7abe200640 1 -- 172.21.15.16:0/4021156616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab80aaff0 con 0x7f7ab80b1270 2024-08-17T02:45:30.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.282+0000 7f7abe200640 1 -- 172.21.15.16:0/4021156616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab806a290 con 0x7f7ab80b1270 2024-08-17T02:45:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.282+0000 7f7abe200640 1 -- 172.21.15.16:0/4021156616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab80aae50 con 0x7f7ab80b1270 2024-08-17T02:45:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.283+0000 7f7aaea00640 1 -- 172.21.15.16:0/4021156616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f7aa8013860 con 0x7f7ab80b1270 2024-08-17T02:45:30.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.284+0000 7f7aaea00640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7a98065150 0x7f7a98067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.284+0000 7f7aaea00640 1 -- 172.21.15.16:0/4021156616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7aa80827d0 con 0x7f7ab80b1270 2024-08-17T02:45:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.285+0000 7f7ab6a00640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7a98065150 0x7f7a98067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:30.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.285+0000 7f7ab6a00640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7a98065150 0x7f7a98067600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4034000 tx=0x7f7aa4032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.288+0000 7f7aaea00640 1 -- 172.21.15.16:0/4021156616 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7ab80aae50 con 0x7f7ab80b1270 2024-08-17T02:45:30.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.471+0000 7f7abe200640 1 -- 172.21.15.16:0/4021156616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ab80aae50 con 0x7f7ab80b1270 2024-08-17T02:45:30.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.472+0000 7f7aaea00640 1 -- 172.21.15.16:0/4021156616 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f7ab80aae50 con 0x7f7ab80b1270 2024-08-17T02:45:30.474 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:30.474 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.474+0000 7f7abe200640 1 -- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7a98065150 msgr2=0x7f7a98067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.474+0000 7f7abe200640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7a98065150 0x7f7a98067600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4034000 tx=0x7f7aa4032000 comp rx=0 tx=0).stop 2024-08-17T02:45:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.474+0000 7f7abe200640 1 -- 172.21.15.16:0/4021156616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab80b1270 msgr2=0x7f7ab813ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.474+0000 7f7abe200640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab80b1270 0x7f7ab813ef50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800ea20 tx=0x7f7aa80090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.474+0000 7f7abe200640 1 -- 172.21.15.16:0/4021156616 shutdown_connections 2024-08-17T02:45:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.474+0000 7f7abe200640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7a98065150 0x7f7a98067600 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.474+0000 7f7abe200640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ab8140580 0x7f7ab8069d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.474+0000 7f7abe200640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ab81401b0 0x7f7ab813f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.474+0000 7f7abe200640 1 --2- 172.21.15.16:0/4021156616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab80b1270 0x7f7ab813ef50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.474+0000 7f7abe200640 1 -- 172.21.15.16:0/4021156616 >> 172.21.15.16:0/4021156616 conn(0x7f7ab80cb9b0 msgr2=0x7f7ab80a54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:30.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.475+0000 7f7abe200640 1 -- 172.21.15.16:0/4021156616 shutdown_connections 2024-08-17T02:45:30.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.475+0000 7f7abe200640 1 -- 172.21.15.16:0/4021156616 wait complete. 2024-08-17T02:45:30.490 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:45:14.291903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:45:18.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:45:18.430718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:45:30.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:45:30.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.642+0000 7f5af9a00640 1 Processor -- start 2024-08-17T02:45:30.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.642+0000 7f5af9a00640 1 -- start start 2024-08-17T02:45:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.643+0000 7f5af9a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5af4155420 0x7f5af4160140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.643+0000 7f5af9a00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af40c83e0 con 0x7f5af406ddb0 2024-08-17T02:45:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.643+0000 7f5af9a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af40cb7d0 con 0x7f5af406d9e0 2024-08-17T02:45:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.643+0000 7f5af9a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40aed90 con 0x7f5af4155420 2024-08-17T02:45:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.643+0000 7f5aebe00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5af406ddb0 0x7f5af4154d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46216/0 (socket says 172.21.15.16:46216) 2024-08-17T02:45:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.643+0000 7f5aebe00640 1 -- 172.21.15.16:0/3917137567 learned_addr learned my addr 172.21.15.16:0/3917137567 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.644+0000 7f5af3400640 1 --2- 172.21.15.16:0/3917137567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5af4155420 0x7f5af4160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.644+0000 7f5af2000640 1 -- 172.21.15.16:0/3917137567 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512170246 0 0) 0x7f5af40c83e0 con 0x7f5af406ddb0 2024-08-17T02:45:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.644+0000 7f5af2000640 1 -- 172.21.15.16:0/3917137567 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ad80034e0 con 0x7f5af406ddb0 2024-08-17T02:45:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.644+0000 7f5af2000640 1 -- 172.21.15.16:0/3917137567 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920982862 0 0) 0x7f5af40cb7d0 con 0x7f5af406d9e0 2024-08-17T02:45:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.644+0000 7f5af2000640 1 -- 172.21.15.16:0/3917137567 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af40c83e0 con 0x7f5af406d9e0 2024-08-17T02:45:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.644+0000 7f5af3400640 1 -- 172.21.15.16:0/3917137567 >> v1:172.21.15.142:6789/0 conn(0x7f5af406d9e0 legacy=0x7f5af40cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.644+0000 7f5af3400640 1 -- 172.21.15.16:0/3917137567 >> v1:172.21.15.16:6789/0 conn(0x7f5af406ddb0 legacy=0x7f5af4154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.644+0000 7f5af3400640 1 -- 172.21.15.16:0/3917137567 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40af070 con 0x7f5af4155420 2024-08-17T02:45:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.645+0000 7f5af3400640 1 --2- 172.21.15.16:0/3917137567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5af4155420 0x7f5af4160140 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00afa0 tx=0x7f5aec00a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4970ba9dea161d9a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.645+0000 7f5af2000640 1 -- 172.21.15.16:0/3917137567 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00a2e0 con 0x7f5af4155420 2024-08-17T02:45:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.646+0000 7f5af2000640 1 -- 172.21.15.16:0/3917137567 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec008450 con 0x7f5af4155420 2024-08-17T02:45:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.646+0000 7f5af2000640 1 -- 172.21.15.16:0/3917137567 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0087f0 con 0x7f5af4155420 2024-08-17T02:45:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.646+0000 7f5af9a00640 1 -- 172.21.15.16:0/3917137567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5af4155420 msgr2=0x7f5af4160140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.646+0000 7f5af9a00640 1 --2- 172.21.15.16:0/3917137567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5af4155420 0x7f5af4160140 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00afa0 tx=0x7f5aec00a890 comp rx=0 tx=0).stop 2024-08-17T02:45:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.646+0000 7f5af9a00640 1 -- 172.21.15.16:0/3917137567 shutdown_connections 2024-08-17T02:45:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.646+0000 7f5af9a00640 1 --2- 172.21.15.16:0/3917137567 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5af4155420 0x7f5af4160140 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.646+0000 7f5af9a00640 1 -- 172.21.15.16:0/3917137567 >> 172.21.15.16:0/3917137567 conn(0x7f5af4067750 msgr2=0x7f5af40cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.646+0000 7f5af9a00640 1 -- 172.21.15.16:0/3917137567 shutdown_connections 2024-08-17T02:45:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.646+0000 7f5af9a00640 1 -- 172.21.15.16:0/3917137567 wait complete. 2024-08-17T02:45:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.648+0000 7f5af9a00640 1 Processor -- start 2024-08-17T02:45:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.648+0000 7f5af9a00640 1 -- start start 2024-08-17T02:45:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.648+0000 7f5af9a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5af406d9e0 0x7f5af40c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af9a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5af406ddb0 0x7f5af40cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af9a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af4155420 0x7f5af40cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af9a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4168db0 con 0x7f5af4155420 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af9a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4169bb0 con 0x7f5af406d9e0 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af9a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4168fa0 con 0x7f5af406ddb0 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af4155420 0x7f5af40cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af4155420 0x7f5af40cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58338/0 (socket says 172.21.15.16:58338) 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af3400640 1 -- 172.21.15.16:0/704459878 learned_addr learned my addr 172.21.15.16:0/704459878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5aebe00640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5af406ddb0 0x7f5af40cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af3400640 1 -- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5af406ddb0 msgr2=0x7f5af40cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af2a00640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5af406d9e0 0x7f5af40c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af3400640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5af406ddb0 0x7f5af40cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af3400640 1 -- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5af406d9e0 msgr2=0x7f5af40c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af3400640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5af406d9e0 0x7f5af40c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af3400640 1 -- 172.21.15.16:0/704459878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec00ab30 con 0x7f5af4155420 2024-08-17T02:45:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5aebe00640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5af406ddb0 0x7f5af40cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af2a00640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5af406d9e0 0x7f5af40c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.649+0000 7f5af3400640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af4155420 0x7f5af40cb550 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00afa0 tx=0x7f5aec035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.650+0000 7f5aea000640 1 -- 172.21.15.16:0/704459878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec00a2e0 con 0x7f5af4155420 2024-08-17T02:45:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.650+0000 7f5aea000640 1 -- 172.21.15.16:0/704459878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec033ab0 con 0x7f5af4155420 2024-08-17T02:45:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.650+0000 7f5aea000640 1 -- 172.21.15.16:0/704459878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec033e50 con 0x7f5af4155420 2024-08-17T02:45:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.650+0000 7f5af9a00640 1 -- 172.21.15.16:0/704459878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af4169280 con 0x7f5af4155420 2024-08-17T02:45:30.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.650+0000 7f5af9a00640 1 -- 172.21.15.16:0/704459878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af4169840 con 0x7f5af4155420 2024-08-17T02:45:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.651+0000 7f5af9a00640 1 -- 172.21.15.16:0/704459878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af4169d60 con 0x7f5af4155420 2024-08-17T02:45:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.652+0000 7f5aea000640 1 -- 172.21.15.16:0/704459878 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f5aec03d420 con 0x7f5af4155420 2024-08-17T02:45:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.653+0000 7f5aea000640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ad0065150 0x7f5ad0067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:30.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.653+0000 7f5aea000640 1 -- 172.21.15.16:0/704459878 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5aec0a2e20 con 0x7f5af4155420 2024-08-17T02:45:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.653+0000 7f5af2a00640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ad0065150 0x7f5ad0067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:30.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.654+0000 7f5af2a00640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ad0065150 0x7f5ad0067600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4007fd0 tx=0x7f5ae4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:30.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.657+0000 7f5aea000640 1 -- 172.21.15.16:0/704459878 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5af4169d60 con 0x7f5af4155420 2024-08-17T02:45:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.839+0000 7f5af9a00640 1 -- 172.21.15.16:0/704459878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5af4169d20 con 0x7f5af4155420 2024-08-17T02:45:30.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.840+0000 7f5aea000640 1 -- 172.21.15.16:0/704459878 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5aec070e40 con 0x7f5af4155420 2024-08-17T02:45:30.841 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-17T02:45:30.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.842+0000 7f5af9a00640 1 -- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ad0065150 msgr2=0x7f5ad0067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.842+0000 7f5af9a00640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ad0065150 0x7f5ad0067600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4007fd0 tx=0x7f5ae4009040 comp rx=0 tx=0).stop 2024-08-17T02:45:30.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.842+0000 7f5af9a00640 1 -- 172.21.15.16:0/704459878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af4155420 msgr2=0x7f5af40cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:30.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.842+0000 7f5af9a00640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af4155420 0x7f5af40cb550 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00afa0 tx=0x7f5aec035000 comp rx=0 tx=0).stop 2024-08-17T02:45:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.842+0000 7f5af9a00640 1 -- 172.21.15.16:0/704459878 shutdown_connections 2024-08-17T02:45:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.842+0000 7f5af9a00640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ad0065150 0x7f5ad0067600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.842+0000 7f5af9a00640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af4155420 0x7f5af40cb550 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.842+0000 7f5af9a00640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5af406ddb0 0x7f5af40cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.842+0000 7f5af9a00640 1 --2- 172.21.15.16:0/704459878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5af406d9e0 0x7f5af40c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.842+0000 7f5af9a00640 1 -- 172.21.15.16:0/704459878 >> 172.21.15.16:0/704459878 conn(0x7f5af4067750 msgr2=0x7f5af4154ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.842+0000 7f5af9a00640 1 -- 172.21.15.16:0/704459878 shutdown_connections 2024-08-17T02:45:30.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:30.842+0000 7f5af9a00640 1 -- 172.21.15.16:0/704459878 wait complete. 2024-08-17T02:45:30.858 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:45:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.010+0000 7f9e8e000640 1 Processor -- start 2024-08-17T02:45:31.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.010+0000 7f9e8e000640 1 -- start start 2024-08-17T02:45:31.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.011+0000 7f9e8e000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e88155a80 0x7f9e8806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.011+0000 7f9e8e000640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e881598f0 con 0x7f9e880c7db0 2024-08-17T02:45:31.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.011+0000 7f9e8e000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e88159bd0 con 0x7f9e880c92e0 2024-08-17T02:45:31.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.011+0000 7f9e8e000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e88159eb0 con 0x7f9e88155a80 2024-08-17T02:45:31.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.011+0000 7f9e87400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e88155a80 0x7f9e8806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.011+0000 7f9e86a00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9e880c7db0 0x7f9e880c8bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46226/0 (socket says 172.21.15.16:46226) 2024-08-17T02:45:31.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.011+0000 7f9e86a00640 1 -- 172.21.15.16:0/3068602593 learned_addr learned my addr 172.21.15.16:0/3068602593 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:31.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.012+0000 7f9e87400640 1 -- 172.21.15.16:0/3068602593 >> v1:172.21.15.142:6789/0 conn(0x7f9e880c92e0 legacy=0x7f9e880aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.012+0000 7f9e87400640 1 -- 172.21.15.16:0/3068602593 >> v1:172.21.15.16:6789/0 conn(0x7f9e880c7db0 legacy=0x7f9e880c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.012+0000 7f9e87400640 1 -- 172.21.15.16:0/3068602593 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8815a190 con 0x7f9e88155a80 2024-08-17T02:45:31.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.012+0000 7f9e86000640 1 -- 172.21.15.16:0/3068602593 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378087281 0 0) 0x7f9e881598f0 con 0x7f9e880c7db0 2024-08-17T02:45:31.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.013+0000 7f9e87400640 1 --2- 172.21.15.16:0/3068602593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e88155a80 0x7f9e8806dc00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009850 tx=0x7f9e780090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c441326497a55859 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:31.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.013+0000 7f9e86000640 1 -- 172.21.15.16:0/3068602593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78032440 con 0x7f9e88155a80 2024-08-17T02:45:31.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.013+0000 7f9e86000640 1 -- 172.21.15.16:0/3068602593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e780325a0 con 0x7f9e88155a80 2024-08-17T02:45:31.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.013+0000 7f9e86000640 1 -- 172.21.15.16:0/3068602593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78032940 con 0x7f9e88155a80 2024-08-17T02:45:31.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.013+0000 7f9e8e000640 1 -- 172.21.15.16:0/3068602593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e88155a80 msgr2=0x7f9e8806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.013+0000 7f9e8e000640 1 --2- 172.21.15.16:0/3068602593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e88155a80 0x7f9e8806dc00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009850 tx=0x7f9e780090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.014+0000 7f9e8e000640 1 -- 172.21.15.16:0/3068602593 shutdown_connections 2024-08-17T02:45:31.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.014+0000 7f9e8e000640 1 --2- 172.21.15.16:0/3068602593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e88155a80 0x7f9e8806dc00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.014+0000 7f9e8e000640 1 -- 172.21.15.16:0/3068602593 >> 172.21.15.16:0/3068602593 conn(0x7f9e88067750 msgr2=0x7f9e880c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:31.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.014+0000 7f9e8e000640 1 -- 172.21.15.16:0/3068602593 shutdown_connections 2024-08-17T02:45:31.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.014+0000 7f9e8e000640 1 -- 172.21.15.16:0/3068602593 wait complete. 2024-08-17T02:45:31.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.015+0000 7f9e8e000640 1 Processor -- start 2024-08-17T02:45:31.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.015+0000 7f9e8e000640 1 -- start start 2024-08-17T02:45:31.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.016+0000 7f9e8e000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e880c7db0 0x7f9e88129e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.016+0000 7f9e8e000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e880c92e0 0x7f9e881580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.016+0000 7f9e8e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e88155a80 0x7f9e880a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.016+0000 7f9e8e000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e88145230 con 0x7f9e880c7db0 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.016+0000 7f9e8e000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e88148850 con 0x7f9e88155a80 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.016+0000 7f9e8e000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e881479a0 con 0x7f9e880c92e0 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.016+0000 7f9e87400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e880c7db0 0x7f9e88129e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.016+0000 7f9e87400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e880c7db0 0x7f9e88129e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58348/0 (socket says 172.21.15.16:58348) 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.016+0000 7f9e87400640 1 -- 172.21.15.16:0/1514971224 learned_addr learned my addr 172.21.15.16:0/1514971224 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e86a00640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e880c92e0 0x7f9e881580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e87400640 1 -- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e880c92e0 msgr2=0x7f9e881580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e87e00640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e88155a80 0x7f9e880a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e87400640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e880c92e0 0x7f9e881580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e87400640 1 -- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e88155a80 msgr2=0x7f9e880a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e87400640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e88155a80 0x7f9e880a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e87400640 1 -- 172.21.15.16:0/1514971224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e78009390 con 0x7f9e880c7db0 2024-08-17T02:45:31.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e86a00640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e880c92e0 0x7f9e881580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:31.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e87e00640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e88155a80 0x7f9e880a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:31.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e87400640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e880c7db0 0x7f9e88129e30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009850 tx=0x7f9e78032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:31.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e7ea00640 1 -- 172.21.15.16:0/1514971224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78031de0 con 0x7f9e880c7db0 2024-08-17T02:45:31.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e7ea00640 1 -- 172.21.15.16:0/1514971224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7803d420 con 0x7f9e880c7db0 2024-08-17T02:45:31.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e8e000640 1 -- 172.21.15.16:0/1514971224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e88147c40 con 0x7f9e880c7db0 2024-08-17T02:45:31.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.017+0000 7f9e7ea00640 1 -- 172.21.15.16:0/1514971224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7803d820 con 0x7f9e880c7db0 2024-08-17T02:45:31.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.018+0000 7f9e8e000640 1 -- 172.21.15.16:0/1514971224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e88148200 con 0x7f9e880c7db0 2024-08-17T02:45:31.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.019+0000 7f9e8e000640 1 -- 172.21.15.16:0/1514971224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e88071050 con 0x7f9e880c7db0 2024-08-17T02:45:31.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.019+0000 7f9e7ea00640 1 -- 172.21.15.16:0/1514971224 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f9e78039040 con 0x7f9e880c7db0 2024-08-17T02:45:31.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.020+0000 7f9e7ea00640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9e68065150 0x7f9e68067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.020+0000 7f9e7ea00640 1 -- 172.21.15.16:0/1514971224 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9e780a2c10 con 0x7f9e880c7db0 2024-08-17T02:45:31.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.021+0000 7f9e86a00640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9e68065150 0x7f9e68067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.022+0000 7f9e86a00640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9e68065150 0x7f9e68067600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e88158c30 tx=0x7f9e74009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:31.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.024+0000 7f9e7ea00640 1 -- 172.21.15.16:0/1514971224 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9e78070cf0 con 0x7f9e880c7db0 2024-08-17T02:45:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.208+0000 7f9e8e000640 1 -- 172.21.15.16:0/1514971224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e8806aa10 con 0x7f9e880c7db0 2024-08-17T02:45:31.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.208+0000 7f9e7ea00640 1 -- 172.21.15.16:0/1514971224 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e78074310 con 0x7f9e880c7db0 2024-08-17T02:45:31.210 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:45:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.211+0000 7f9e8e000640 1 -- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9e68065150 msgr2=0x7f9e68067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.211+0000 7f9e8e000640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9e68065150 0x7f9e68067600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9e88158c30 tx=0x7f9e74009040 comp rx=0 tx=0).stop 2024-08-17T02:45:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.211+0000 7f9e8e000640 1 -- 172.21.15.16:0/1514971224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e880c7db0 msgr2=0x7f9e88129e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.211+0000 7f9e8e000640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e880c7db0 0x7f9e88129e30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009850 tx=0x7f9e78032000 comp rx=0 tx=0).stop 2024-08-17T02:45:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.211+0000 7f9e8e000640 1 -- 172.21.15.16:0/1514971224 shutdown_connections 2024-08-17T02:45:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.211+0000 7f9e8e000640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9e68065150 0x7f9e68067600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.211+0000 7f9e8e000640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e88155a80 0x7f9e880a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.211+0000 7f9e8e000640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9e880c92e0 0x7f9e881580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.211+0000 7f9e8e000640 1 --2- 172.21.15.16:0/1514971224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e880c7db0 0x7f9e88129e30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.211+0000 7f9e8e000640 1 -- 172.21.15.16:0/1514971224 >> 172.21.15.16:0/1514971224 conn(0x7f9e88067750 msgr2=0x7f9e880cb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.211+0000 7f9e8e000640 1 -- 172.21.15.16:0/1514971224 shutdown_connections 2024-08-17T02:45:31.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.211+0000 7f9e8e000640 1 -- 172.21.15.16:0/1514971224 wait complete. 2024-08-17T02:45:31.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:45:31.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.379+0000 7f665de00640 1 Processor -- start 2024-08-17T02:45:31.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.380+0000 7f665de00640 1 -- start start 2024-08-17T02:45:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.381+0000 7f665de00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6658069230 0x7f6658069600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.381+0000 7f665de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6658096e70 0x7f66580abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.381+0000 7f665de00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66580ac4f0 con 0x7f6658067de0 2024-08-17T02:45:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.381+0000 7f665de00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66580ad010 con 0x7f6658069230 2024-08-17T02:45:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.381+0000 7f665de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66580ae3a0 con 0x7f6658096e70 2024-08-17T02:45:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.381+0000 7f665ca00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6658069230 0x7f6658069600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.381+0000 7f665d400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6658067de0 0x7f6658068200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46242/0 (socket says 172.21.15.16:46242) 2024-08-17T02:45:31.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.381+0000 7f665d400640 1 -- 172.21.15.16:0/1041736606 learned_addr learned my addr 172.21.15.16:0/1041736606 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:31.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.381+0000 7f6657e00640 1 --2- 172.21.15.16:0/1041736606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6658096e70 0x7f66580abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.382+0000 7f665ca00640 1 -- 172.21.15.16:0/1041736606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6658096e70 msgr2=0x7f66580abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.382+0000 7f665ca00640 1 --2- 172.21.15.16:0/1041736606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6658096e70 0x7f66580abfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.382+0000 7f665ca00640 1 -- 172.21.15.16:0/1041736606 >> v1:172.21.15.16:6789/0 conn(0x7f6658067de0 legacy=0x7f6658068200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.382+0000 7f6657400640 1 -- 172.21.15.16:0/1041736606 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521752382 0 0) 0x7f66580ac4f0 con 0x7f6658067de0 2024-08-17T02:45:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.382+0000 7f665ca00640 1 -- 172.21.15.16:0/1041736606 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66580ac770 con 0x7f6658069230 2024-08-17T02:45:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.382+0000 7f6657e00640 1 --2- 172.21.15.16:0/1041736606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6658096e70 0x7f66580abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.383+0000 7f665ca00640 1 --2- 172.21.15.16:0/1041736606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6658069230 0x7f6658069600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f664c009850 tx=0x7f664c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b291b6b42ec0837 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.383+0000 7f6657400640 1 -- 172.21.15.16:0/1041736606 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c02ac80 con 0x7f6658069230 2024-08-17T02:45:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.383+0000 7f6657400640 1 -- 172.21.15.16:0/1041736606 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c02ade0 con 0x7f6658069230 2024-08-17T02:45:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.383+0000 7f6657400640 1 -- 172.21.15.16:0/1041736606 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c032670 con 0x7f6658069230 2024-08-17T02:45:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.383+0000 7f665de00640 1 -- 172.21.15.16:0/1041736606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6658069230 msgr2=0x7f6658069600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.383+0000 7f665de00640 1 --2- 172.21.15.16:0/1041736606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6658069230 0x7f6658069600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f664c009850 tx=0x7f664c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.384+0000 7f665de00640 1 -- 172.21.15.16:0/1041736606 shutdown_connections 2024-08-17T02:45:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.384+0000 7f665de00640 1 --2- 172.21.15.16:0/1041736606 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6658096e70 0x7f66580abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.384+0000 7f665de00640 1 --2- 172.21.15.16:0/1041736606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6658069230 0x7f6658069600 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.384+0000 7f665de00640 1 -- 172.21.15.16:0/1041736606 >> 172.21.15.16:0/1041736606 conn(0x7f6658096a70 msgr2=0x7f66580cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.384+0000 7f665de00640 1 -- 172.21.15.16:0/1041736606 shutdown_connections 2024-08-17T02:45:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.384+0000 7f665de00640 1 -- 172.21.15.16:0/1041736606 wait complete. 2024-08-17T02:45:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.385+0000 7f665de00640 1 Processor -- start 2024-08-17T02:45:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.385+0000 7f665de00640 1 -- start start 2024-08-17T02:45:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.386+0000 7f665de00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6658067de0 0x7f66581548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.386+0000 7f665de00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6658069230 0x7f6658150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.386+0000 7f665de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658096e70 0x7f665814d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.386+0000 7f665de00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6658150ea0 con 0x7f6658069230 2024-08-17T02:45:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.386+0000 7f665de00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6658154fa0 con 0x7f6658096e70 2024-08-17T02:45:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.386+0000 7f665de00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665814df40 con 0x7f6658067de0 2024-08-17T02:45:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.386+0000 7f6657e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6658069230 0x7f6658150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.386+0000 7f6657e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6658069230 0x7f6658150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58364/0 (socket says 172.21.15.16:58364) 2024-08-17T02:45:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.386+0000 7f6657e00640 1 -- 172.21.15.16:0/2187303343 learned_addr learned my addr 172.21.15.16:0/2187303343 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.386+0000 7f665ca00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6658067de0 0x7f66581548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.386+0000 7f665d400640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658096e70 0x7f665814d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.387+0000 7f6657e00640 1 -- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6658067de0 msgr2=0x7f66581548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.387+0000 7f6657e00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6658067de0 0x7f66581548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.387+0000 7f6657e00640 1 -- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658096e70 msgr2=0x7f665814d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.387+0000 7f6657e00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658096e70 0x7f665814d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.387+0000 7f6657e00640 1 -- 172.21.15.16:0/2187303343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c009390 con 0x7f6658069230 2024-08-17T02:45:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.387+0000 7f665d400640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658096e70 0x7f665814d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.387+0000 7f6657e00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6658069230 0x7f6658150960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f664400c880 tx=0x7f6644008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.387+0000 7f664be00640 1 -- 172.21.15.16:0/2187303343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664400ce10 con 0x7f6658069230 2024-08-17T02:45:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.387+0000 7f664be00640 1 -- 172.21.15.16:0/2187303343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6644007830 con 0x7f6658069230 2024-08-17T02:45:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.387+0000 7f664be00640 1 -- 172.21.15.16:0/2187303343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644007c10 con 0x7f6658069230 2024-08-17T02:45:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.387+0000 7f665de00640 1 -- 172.21.15.16:0/2187303343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665814e220 con 0x7f6658069230 2024-08-17T02:45:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.387+0000 7f665de00640 1 -- 172.21.15.16:0/2187303343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665814e7e0 con 0x7f6658069230 2024-08-17T02:45:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.388+0000 7f665de00640 1 -- 172.21.15.16:0/2187303343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6658071050 con 0x7f6658069230 2024-08-17T02:45:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.389+0000 7f664be00640 1 -- 172.21.15.16:0/2187303343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f6644011600 con 0x7f6658069230 2024-08-17T02:45:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.390+0000 7f664be00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6638065150 0x7f6638067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.390+0000 7f664be00640 1 -- 172.21.15.16:0/2187303343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6644082750 con 0x7f6658069230 2024-08-17T02:45:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.391+0000 7f665ca00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6638065150 0x7f6638067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.392+0000 7f665ca00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6638065150 0x7f6638067600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f664c034000 tx=0x7f664c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:31.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.395+0000 7f664be00640 1 -- 172.21.15.16:0/2187303343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6644050820 con 0x7f6658069230 2024-08-17T02:45:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.581+0000 7f665de00640 1 -- 172.21.15.16:0/2187303343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f66580cd5e0 con 0x7f6658069230 2024-08-17T02:45:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.581+0000 7f664be00640 1 -- 172.21.15.16:0/2187303343 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6644053e40 con 0x7f6658069230 2024-08-17T02:45:31.583 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-08-17T02:45:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.584+0000 7f665de00640 1 -- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6638065150 msgr2=0x7f6638067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.584+0000 7f665de00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6638065150 0x7f6638067600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f664c034000 tx=0x7f664c032000 comp rx=0 tx=0).stop 2024-08-17T02:45:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.584+0000 7f665de00640 1 -- 172.21.15.16:0/2187303343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6658069230 msgr2=0x7f6658150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.584+0000 7f665de00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6658069230 0x7f6658150960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f664400c880 tx=0x7f6644008040 comp rx=0 tx=0).stop 2024-08-17T02:45:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.584+0000 7f665de00640 1 -- 172.21.15.16:0/2187303343 shutdown_connections 2024-08-17T02:45:31.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.584+0000 7f665de00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6638065150 0x7f6638067600 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.584+0000 7f665de00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658096e70 0x7f665814d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.584+0000 7f665de00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6658069230 0x7f6658150960 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.584+0000 7f665de00640 1 --2- 172.21.15.16:0/2187303343 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6658067de0 0x7f66581548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.584+0000 7f665de00640 1 -- 172.21.15.16:0/2187303343 >> 172.21.15.16:0/2187303343 conn(0x7f6658096a70 msgr2=0x7f665806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:31.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.584+0000 7f665de00640 1 -- 172.21.15.16:0/2187303343 shutdown_connections 2024-08-17T02:45:31.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.584+0000 7f665de00640 1 -- 172.21.15.16:0/2187303343 wait complete. 2024-08-17T02:45:31.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:45:31.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.752+0000 7fbb6bc00640 1 Processor -- start 2024-08-17T02:45:31.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.752+0000 7fbb6bc00640 1 -- start start 2024-08-17T02:45:31.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.753+0000 7fbb6bc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbb640ada10 0x7fbb640b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.753+0000 7fbb6bc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb640b38b0 0x7fbb640aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.753+0000 7fbb6bc00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb640d0ab0 con 0x7fbb640b1120 2024-08-17T02:45:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.753+0000 7fbb6bc00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb640b00c0 con 0x7fbb640ada10 2024-08-17T02:45:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.753+0000 7fbb6bc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640af330 con 0x7fbb640b38b0 2024-08-17T02:45:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.754+0000 7fbb63e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbb640b1120 0x7fbb640b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46246/0 (socket says 172.21.15.16:46246) 2024-08-17T02:45:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.754+0000 7fbb63e00640 1 -- 172.21.15.16:0/2385946714 learned_addr learned my addr 172.21.15.16:0/2385946714 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.754+0000 7fbb69200640 1 --2- 172.21.15.16:0/2385946714 >> v2:172.21.15.142:3300/0 conn(0x7fbb640ada10 0x7fbb640b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.754+0000 7fbb69c00640 1 --2- 172.21.15.16:0/2385946714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb640b38b0 0x7fbb640aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.754+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319360209 0 0) 0x7fbb640d0ab0 con 0x7fbb640b1120 2024-08-17T02:45:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.754+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb4c0034e0 con 0x7fbb640b1120 2024-08-17T02:45:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.754+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 675134631 0 0) 0x7fbb4c0034e0 con 0x7fbb640b1120 2024-08-17T02:45:31.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.754+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb640b38b0 msgr2=0x7fbb640aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.754+0000 7fbb63400640 1 --2- 172.21.15.16:0/2385946714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb640b38b0 0x7fbb640aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.754+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 >> v2:172.21.15.142:3300/0 conn(0x7fbb640ada10 msgr2=0x7fbb640b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.754+0000 7fbb63400640 1 --2- 172.21.15.16:0/2385946714 >> v2:172.21.15.142:3300/0 conn(0x7fbb640ada10 0x7fbb640b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.754+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640b0360 con 0x7fbb640b1120 2024-08-17T02:45:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7fbb50001470 con 0x7fbb640b1120 2024-08-17T02:45:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb69200640 1 --2- 172.21.15.16:0/2385946714 >> v2:172.21.15.142:3300/0 conn(0x7fbb640ada10 0x7fbb640b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 >> v1:172.21.15.16:6789/0 conn(0x7fbb640b1120 legacy=0x7fbb640b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb63400640 1 --2- 172.21.15.16:0/2385946714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c003a50 0x7fbb4c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb63400640 1 --2- 172.21.15.16:0/2385946714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb640b38b0 0x7fbb4c004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb63400640 1 --2- 172.21.15.16:0/2385946714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb4c004cc0 0x7fbb4c005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640b00c0 con 0x7fbb4c003a50 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640af330 con 0x7fbb4c004cc0 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c00bab0 con 0x7fbb640b38b0 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb69200640 1 --2- 172.21.15.16:0/2385946714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c003a50 0x7fbb4c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb63e00640 1 --2- 172.21.15.16:0/2385946714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb640b38b0 0x7fbb4c004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb69c00640 1 --2- 172.21.15.16:0/2385946714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb4c004cc0 0x7fbb4c005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb69200640 1 -- 172.21.15.16:0/2385946714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb640b38b0 msgr2=0x7fbb4c004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb69200640 1 --2- 172.21.15.16:0/2385946714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb640b38b0 0x7fbb4c004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb69200640 1 -- 172.21.15.16:0/2385946714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb4c004cc0 msgr2=0x7fbb4c005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb69200640 1 --2- 172.21.15.16:0/2385946714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb4c004cc0 0x7fbb4c005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.755+0000 7fbb69200640 1 -- 172.21.15.16:0/2385946714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c0034e0 con 0x7fbb4c003a50 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.756+0000 7fbb69200640 1 --2- 172.21.15.16:0/2385946714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c003a50 0x7fbb4c003e60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009d60 tx=0x7fbb580090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64056583ac83515a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:31.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.757+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58031400 con 0x7fbb4c003a50 2024-08-17T02:45:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.757+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5802d050 con 0x7fbb4c003a50 2024-08-17T02:45:31.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.757+0000 7fbb63400640 1 -- 172.21.15.16:0/2385946714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58031770 con 0x7fbb4c003a50 2024-08-17T02:45:31.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.757+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2385946714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c003a50 msgr2=0x7fbb4c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.757+0000 7fbb6bc00640 1 --2- 172.21.15.16:0/2385946714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c003a50 0x7fbb4c003e60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009d60 tx=0x7fbb580090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.757+0000 7fbb69c00640 1 -- 172.21.15.16:0/2385946714 reap_dead start 2024-08-17T02:45:31.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.757+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2385946714 shutdown_connections 2024-08-17T02:45:31.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.757+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2385946714 >> 172.21.15.16:0/2385946714 conn(0x7fbb640cfdf0 msgr2=0x7fbb640d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:31.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.757+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2385946714 shutdown_connections 2024-08-17T02:45:31.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.757+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2385946714 wait complete. 2024-08-17T02:45:31.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.759+0000 7fbb6bc00640 1 Processor -- start 2024-08-17T02:45:31.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.759+0000 7fbb6bc00640 1 -- start start 2024-08-17T02:45:31.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.759+0000 7fbb6bc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb4c003a50 0x7fbb64153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb6bc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159a30 0x7fbb6409d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb6bc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6415a3c0 0x7fbb6415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb6bc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6409cf00 con 0x7fbb6415a3c0 2024-08-17T02:45:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb6bc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb64153750 con 0x7fbb4c003a50 2024-08-17T02:45:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb6bc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6409c2f0 con 0x7fbb64159a30 2024-08-17T02:45:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb69c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6415a3c0 0x7fbb6415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb69c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6415a3c0 0x7fbb6415a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58390/0 (socket says 172.21.15.16:58390) 2024-08-17T02:45:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb69c00640 1 -- 172.21.15.16:0/2755035854 learned_addr learned my addr 172.21.15.16:0/2755035854 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb69200640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb4c003a50 0x7fbb64153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb63e00640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159a30 0x7fbb6409d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb69c00640 1 -- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159a30 msgr2=0x7fbb6409d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb69c00640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159a30 0x7fbb6409d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb69c00640 1 -- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb4c003a50 msgr2=0x7fbb64153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb69c00640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb4c003a50 0x7fbb64153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.760+0000 7fbb69c00640 1 -- 172.21.15.16:0/2755035854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6409c5d0 con 0x7fbb6415a3c0 2024-08-17T02:45:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.761+0000 7fbb69c00640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6415a3c0 0x7fbb6415a860 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0094d0 tx=0x7fbb5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.761+0000 7fbb57e00640 1 -- 172.21.15.16:0/2755035854 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c00e040 con 0x7fbb6415a3c0 2024-08-17T02:45:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.761+0000 7fbb57e00640 1 -- 172.21.15.16:0/2755035854 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c00f040 con 0x7fbb6415a3c0 2024-08-17T02:45:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.761+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2755035854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6409c8b0 con 0x7fbb6415a3c0 2024-08-17T02:45:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.761+0000 7fbb57e00640 1 -- 172.21.15.16:0/2755035854 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c015920 con 0x7fbb6415a3c0 2024-08-17T02:45:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.761+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2755035854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6415ada0 con 0x7fbb6415a3c0 2024-08-17T02:45:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.762+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2755035854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6409d070 con 0x7fbb6415a3c0 2024-08-17T02:45:31.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.763+0000 7fbb57e00640 1 -- 172.21.15.16:0/2755035854 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fbb5c015d10 con 0x7fbb6415a3c0 2024-08-17T02:45:31.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.764+0000 7fbb57e00640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbb44065150 0x7fbb44067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:31.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.764+0000 7fbb57e00640 1 -- 172.21.15.16:0/2755035854 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbb5c082510 con 0x7fbb6415a3c0 2024-08-17T02:45:31.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.764+0000 7fbb69200640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbb44065150 0x7fbb44067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:31.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.765+0000 7fbb69200640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbb44065150 0x7fbb44067600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbb58034000 tx=0x7fbb58032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:31.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.768+0000 7fbb57e00640 1 -- 172.21.15.16:0/2755035854 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fbb5c054070 con 0x7fbb6415a3c0 2024-08-17T02:45:31.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.987+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2755035854 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb64064db0 con 0x7fbb6415a3c0 2024-08-17T02:45:31.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.988+0000 7fbb57e00640 1 -- 172.21.15.16:0/2755035854 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7fbb5c053c00 con 0x7fbb6415a3c0 2024-08-17T02:45:31.989 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:31.989 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/111049205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":111049205},{"type":"v1","addr":"172.21.15.142:6836","nonce":111049205}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2692359383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2692359383},{"type":"v1","addr":"172.21.15.142:6837","nonce":2692359383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3179355872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3179355872},{"type":"v1","addr":"172.21.15.16:6833","nonce":3179355872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:45:19.450658+0000","modified":"2024-08-17T02:45:23.039406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2140506421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2140506421},{"type":"v1","addr":"172.21.15.16:6835","nonce":2140506421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:45:31.989 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-08-17T02:45:31.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.989+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbb44065150 msgr2=0x7fbb44067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.989+0000 7fbb6bc00640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbb44065150 0x7fbb44067600 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbb58034000 tx=0x7fbb58032000 comp rx=0 tx=0).stop 2024-08-17T02:45:31.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.989+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2755035854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6415a3c0 msgr2=0x7fbb6415a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:31.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.989+0000 7fbb6bc00640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6415a3c0 0x7fbb6415a860 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0094d0 tx=0x7fbb5c00b040 comp rx=0 tx=0).stop 2024-08-17T02:45:31.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.990+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2755035854 shutdown_connections 2024-08-17T02:45:31.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.990+0000 7fbb6bc00640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbb44065150 0x7fbb44067600 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.990+0000 7fbb6bc00640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb6415a3c0 0x7fbb6415a860 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.990+0000 7fbb6bc00640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbb64159a30 0x7fbb6409d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.990+0000 7fbb6bc00640 1 --2- 172.21.15.16:0/2755035854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb4c003a50 0x7fbb64153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:31.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.990+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2755035854 >> 172.21.15.16:0/2755035854 conn(0x7fbb640cfdf0 msgr2=0x7fbb640b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:31.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.990+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2755035854 shutdown_connections 2024-08-17T02:45:31.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:31.990+0000 7fbb6bc00640 1 -- 172.21.15.16:0/2755035854 wait complete. 2024-08-17T02:45:32.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.156+0000 7f0ff0a00640 1 Processor -- start 2024-08-17T02:45:32.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.156+0000 7f0ff0a00640 1 -- start start 2024-08-17T02:45:32.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.157+0000 7f0ff0a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0fec0caa40 0x7f0fec0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.157+0000 7f0ff0a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fec159350 0x7f0fec0ad380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.157+0000 7f0ff0a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fec0cb560 con 0x7f0fec158f80 2024-08-17T02:45:32.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.157+0000 7f0ff0a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fec0cf7e0 con 0x7f0fec0caa40 2024-08-17T02:45:32.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.157+0000 7f0ff0a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0cfac0 con 0x7f0fec159350 2024-08-17T02:45:32.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.158+0000 7f0fe9400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0fec0caa40 0x7f0fec0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.158+0000 7f0fe9e00640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f0fec158f80 0x7f0fec0ca330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:35102/0 (socket says 172.21.15.16:35102) 2024-08-17T02:45:32.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.158+0000 7f0fe9e00640 1 -- 172.21.15.16:0/116975878 learned_addr learned my addr 172.21.15.16:0/116975878 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:32.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.158+0000 7f0fea800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fec159350 0x7f0fec0ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.158+0000 7f0fe8a00640 1 -- 172.21.15.16:0/116975878 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029669255 0 0) 0x7f0fec0cb560 con 0x7f0fec158f80 2024-08-17T02:45:32.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.158+0000 7f0fe8a00640 1 -- 172.21.15.16:0/116975878 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd00034e0 con 0x7f0fec158f80 2024-08-17T02:45:32.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.158+0000 7f0fea800640 1 -- 172.21.15.16:0/116975878 >> v2:172.21.15.142:3300/0 conn(0x7f0fec0caa40 msgr2=0x7f0fec0ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:32.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.158+0000 7f0fea800640 1 --2- 172.21.15.16:0/116975878 >> v2:172.21.15.142:3300/0 conn(0x7f0fec0caa40 0x7f0fec0ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.158+0000 7f0fea800640 1 -- 172.21.15.16:0/116975878 >> v1:172.21.15.142:6789/0 conn(0x7f0fec158f80 legacy=0x7f0fec0ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.158+0000 7f0fea800640 1 -- 172.21.15.16:0/116975878 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec0caf10 con 0x7f0fec159350 2024-08-17T02:45:32.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.158+0000 7f0fe9400640 1 --2- 172.21.15.16:0/116975878 >> v2:172.21.15.142:3300/0 conn(0x7f0fec0caa40 0x7f0fec0ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:32.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.159+0000 7f0fea800640 1 --2- 172.21.15.16:0/116975878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fec159350 0x7f0fec0ad380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009e60 tx=0x7f0fe4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d3df4e4883c13f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:32.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.160+0000 7f0fe8a00640 1 -- 172.21.15.16:0/116975878 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4007550 con 0x7f0fec159350 2024-08-17T02:45:32.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.160+0000 7f0fe8a00640 1 -- 172.21.15.16:0/116975878 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe402d050 con 0x7f0fec159350 2024-08-17T02:45:32.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.160+0000 7f0fe8a00640 1 -- 172.21.15.16:0/116975878 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4007920 con 0x7f0fec159350 2024-08-17T02:45:32.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.160+0000 7f0ff0a00640 1 -- 172.21.15.16:0/116975878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fec159350 msgr2=0x7f0fec0ad380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.160+0000 7f0ff0a00640 1 --2- 172.21.15.16:0/116975878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fec159350 0x7f0fec0ad380 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009e60 tx=0x7f0fe4009780 comp rx=0 tx=0).stop 2024-08-17T02:45:32.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.160+0000 7f0ff0a00640 1 -- 172.21.15.16:0/116975878 shutdown_connections 2024-08-17T02:45:32.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.160+0000 7f0ff0a00640 1 --2- 172.21.15.16:0/116975878 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fec159350 0x7f0fec0ad380 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.160+0000 7f0ff0a00640 1 --2- 172.21.15.16:0/116975878 >> v2:172.21.15.142:3300/0 conn(0x7f0fec0caa40 0x7f0fec0ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.160+0000 7f0ff0a00640 1 -- 172.21.15.16:0/116975878 >> 172.21.15.16:0/116975878 conn(0x7f0fec0cfd90 msgr2=0x7f0fec0d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:32.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.160+0000 7f0ff0a00640 1 -- 172.21.15.16:0/116975878 shutdown_connections 2024-08-17T02:45:32.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.160+0000 7f0ff0a00640 1 -- 172.21.15.16:0/116975878 wait complete. 2024-08-17T02:45:32.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.162+0000 7f0ff0a00640 1 Processor -- start 2024-08-17T02:45:32.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.162+0000 7f0ff0a00640 1 -- start start 2024-08-17T02:45:32.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.163+0000 7f0ff0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fec0caa40 0x7f0fec159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.163+0000 7f0ff0a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fec158f80 0x7f0fec154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.163+0000 7f0ff0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec159350 0x7f0fec15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.163+0000 7f0ff0a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec0b37d0 con 0x7f0fec159350 2024-08-17T02:45:32.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.163+0000 7f0ff0a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec159eb0 con 0x7f0fec0caa40 2024-08-17T02:45:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.163+0000 7f0ff0a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec155140 con 0x7f0fec158f80 2024-08-17T02:45:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.163+0000 7f0fea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec159350 0x7f0fec15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.163+0000 7f0fea800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec159350 0x7f0fec15dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58406/0 (socket says 172.21.15.16:58406) 2024-08-17T02:45:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.163+0000 7f0fea800640 1 -- 172.21.15.16:0/3117297932 learned_addr learned my addr 172.21.15.16:0/3117297932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.163+0000 7f0fe9400640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fec158f80 0x7f0fec154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.163+0000 7f0fe9e00640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fec0caa40 0x7f0fec159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.164+0000 7f0fea800640 1 -- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fec158f80 msgr2=0x7f0fec154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.164+0000 7f0fea800640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fec158f80 0x7f0fec154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.164+0000 7f0fea800640 1 -- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fec0caa40 msgr2=0x7f0fec159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.164+0000 7f0fea800640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fec0caa40 0x7f0fec159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.164+0000 7f0fea800640 1 -- 172.21.15.16:0/3117297932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd8009390 con 0x7f0fec159350 2024-08-17T02:45:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.164+0000 7f0fe9400640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fec158f80 0x7f0fec154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.164+0000 7f0fea800640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec159350 0x7f0fec15dc00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0fe402ac90 tx=0x7f0fe4037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.164+0000 7f0fe1600640 1 -- 172.21.15.16:0/3117297932 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4031890 con 0x7f0fec159350 2024-08-17T02:45:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.164+0000 7f0fe1600640 1 -- 172.21.15.16:0/3117297932 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe4040070 con 0x7f0fec159350 2024-08-17T02:45:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.164+0000 7f0ff0a00640 1 -- 172.21.15.16:0/3117297932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe4009a20 con 0x7f0fec159350 2024-08-17T02:45:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.164+0000 7f0fe1600640 1 -- 172.21.15.16:0/3117297932 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4031c40 con 0x7f0fec159350 2024-08-17T02:45:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.164+0000 7f0ff0a00640 1 -- 172.21.15.16:0/3117297932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec15c6a0 con 0x7f0fec159350 2024-08-17T02:45:32.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.165+0000 7f0ff0a00640 1 -- 172.21.15.16:0/3117297932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec0b3910 con 0x7f0fec159350 2024-08-17T02:45:32.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.166+0000 7f0fe1600640 1 -- 172.21.15.16:0/3117297932 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f0fe4042030 con 0x7f0fec159350 2024-08-17T02:45:32.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.167+0000 7f0fe1600640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0fcc065150 0x7f0fcc067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.167+0000 7f0fe1600640 1 -- 172.21.15.16:0/3117297932 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0fe40a3b30 con 0x7f0fec159350 2024-08-17T02:45:32.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.167+0000 7f0fe9e00640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0fcc065150 0x7f0fcc067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.168+0000 7f0fe9e00640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0fcc065150 0x7f0fcc067600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc005840 tx=0x7f0fdc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:32.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.171+0000 7f0fe1600640 1 -- 172.21.15.16:0/3117297932 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0fec0b3910 con 0x7f0fec159350 2024-08-17T02:45:32.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.354+0000 7f0ff0a00640 1 -- 172.21.15.16:0/3117297932 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fec0b3910 con 0x7f0fec159350 2024-08-17T02:45:32.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.355+0000 7f0fe1600640 1 -- 172.21.15.16:0/3117297932 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f0fec0b3910 con 0x7f0fec159350 2024-08-17T02:45:32.359 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:32.359 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:32.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.357+0000 7f0ff0a00640 1 -- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0fcc065150 msgr2=0x7f0fcc067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.357+0000 7f0ff0a00640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0fcc065150 0x7f0fcc067600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc005840 tx=0x7f0fdc009040 comp rx=0 tx=0).stop 2024-08-17T02:45:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.357+0000 7f0ff0a00640 1 -- 172.21.15.16:0/3117297932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec159350 msgr2=0x7f0fec15dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.357+0000 7f0ff0a00640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec159350 0x7f0fec15dc00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0fe402ac90 tx=0x7f0fe4037000 comp rx=0 tx=0).stop 2024-08-17T02:45:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.357+0000 7f0ff0a00640 1 -- 172.21.15.16:0/3117297932 shutdown_connections 2024-08-17T02:45:32.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.357+0000 7f0ff0a00640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0fcc065150 0x7f0fcc067600 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.357+0000 7f0ff0a00640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fec159350 0x7f0fec15dc00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.357+0000 7f0ff0a00640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0fec158f80 0x7f0fec154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.357+0000 7f0ff0a00640 1 --2- 172.21.15.16:0/3117297932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fec0caa40 0x7f0fec159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.357+0000 7f0ff0a00640 1 -- 172.21.15.16:0/3117297932 >> 172.21.15.16:0/3117297932 conn(0x7f0fec0cfd90 msgr2=0x7f0fec0d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.357+0000 7f0ff0a00640 1 -- 172.21.15.16:0/3117297932 shutdown_connections 2024-08-17T02:45:32.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.357+0000 7f0ff0a00640 1 -- 172.21.15.16:0/3117297932 wait complete. 2024-08-17T02:45:32.373 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:45:14.291903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:45:18.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:45:18.430718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:45:32.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:45:32.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.526+0000 7fe742e00640 1 Processor -- start 2024-08-17T02:45:32.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.526+0000 7fe742e00640 1 -- start start 2024-08-17T02:45:32.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.527+0000 7fe742e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe73c0b0a80 0x7fe73c0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.527+0000 7fe742e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe73c157e10 0x7fe73c0ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.527+0000 7fe742e00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0ae150 con 0x7fe73c0adcf0 2024-08-17T02:45:32.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.527+0000 7fe742e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c0b1390 con 0x7fe73c157e10 2024-08-17T02:45:32.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.527+0000 7fe742e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c157070 con 0x7fe73c0b0a80 2024-08-17T02:45:32.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.528+0000 7fe740e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe73c0adcf0 0x7fe73c15a6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46250/0 (socket says 172.21.15.16:46250) 2024-08-17T02:45:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.528+0000 7fe740e00640 1 -- 172.21.15.16:0/695249148 learned_addr learned my addr 172.21.15.16:0/695249148 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.528+0000 7fe73be00640 1 --2- 172.21.15.16:0/695249148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe73c0b0a80 0x7fe73c0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.528+0000 7fe733e00640 1 --2- 172.21.15.16:0/695249148 >> v2:172.21.15.142:3300/0 conn(0x7fe73c157e10 0x7fe73c0ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.528+0000 7fe73b400640 1 -- 172.21.15.16:0/695249148 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683714498 0 0) 0x7fe734001490 con 0x7fe73c0adcf0 2024-08-17T02:45:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.528+0000 7fe73b400640 1 -- 172.21.15.16:0/695249148 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7200034e0 con 0x7fe73c0adcf0 2024-08-17T02:45:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.528+0000 7fe73be00640 1 -- 172.21.15.16:0/695249148 >> v2:172.21.15.142:3300/0 conn(0x7fe73c157e10 msgr2=0x7fe73c0ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.528+0000 7fe73be00640 1 --2- 172.21.15.16:0/695249148 >> v2:172.21.15.142:3300/0 conn(0x7fe73c157e10 0x7fe73c0ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.528+0000 7fe73be00640 1 -- 172.21.15.16:0/695249148 >> v1:172.21.15.16:6789/0 conn(0x7fe73c0adcf0 legacy=0x7fe73c15a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.528+0000 7fe73be00640 1 -- 172.21.15.16:0/695249148 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c15b410 con 0x7fe73c0b0a80 2024-08-17T02:45:32.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.528+0000 7fe733e00640 1 --2- 172.21.15.16:0/695249148 >> v2:172.21.15.142:3300/0 conn(0x7fe73c157e10 0x7fe73c0ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:32.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.529+0000 7fe73be00640 1 --2- 172.21.15.16:0/695249148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe73c0b0a80 0x7fe73c0b0e50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009850 tx=0x7fe72c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a6fdca742409039 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.530+0000 7fe73b400640 1 -- 172.21.15.16:0/695249148 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c02ad00 con 0x7fe73c0b0a80 2024-08-17T02:45:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.530+0000 7fe73b400640 1 -- 172.21.15.16:0/695249148 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c02ae60 con 0x7fe73c0b0a80 2024-08-17T02:45:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.530+0000 7fe73b400640 1 -- 172.21.15.16:0/695249148 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c032650 con 0x7fe73c0b0a80 2024-08-17T02:45:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.530+0000 7fe742e00640 1 -- 172.21.15.16:0/695249148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe73c0b0a80 msgr2=0x7fe73c0b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.530+0000 7fe742e00640 1 --2- 172.21.15.16:0/695249148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe73c0b0a80 0x7fe73c0b0e50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009850 tx=0x7fe72c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.530+0000 7fe742e00640 1 -- 172.21.15.16:0/695249148 shutdown_connections 2024-08-17T02:45:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.530+0000 7fe742e00640 1 --2- 172.21.15.16:0/695249148 >> v2:172.21.15.142:3300/0 conn(0x7fe73c157e10 0x7fe73c0ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.530+0000 7fe742e00640 1 --2- 172.21.15.16:0/695249148 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe73c0b0a80 0x7fe73c0b0e50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.530+0000 7fe742e00640 1 -- 172.21.15.16:0/695249148 >> 172.21.15.16:0/695249148 conn(0x7fe73c067750 msgr2=0x7fe73c0cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.531+0000 7fe742e00640 1 -- 172.21.15.16:0/695249148 shutdown_connections 2024-08-17T02:45:32.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.531+0000 7fe742e00640 1 -- 172.21.15.16:0/695249148 wait complete. 2024-08-17T02:45:32.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.532+0000 7fe742e00640 1 Processor -- start 2024-08-17T02:45:32.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.532+0000 7fe742e00640 1 -- start start 2024-08-17T02:45:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.533+0000 7fe742e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73c0adcf0 0x7fe73c156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.533+0000 7fe742e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe73c0b0a80 0x7fe73c16a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.533+0000 7fe742e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe73c157e10 0x7fe73c16aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.533+0000 7fe742e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c13ebe0 con 0x7fe73c0adcf0 2024-08-17T02:45:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.533+0000 7fe742e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c13f480 con 0x7fe73c157e10 2024-08-17T02:45:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.533+0000 7fe742e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c14fe60 con 0x7fe73c0b0a80 2024-08-17T02:45:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.533+0000 7fe73be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73c0adcf0 0x7fe73c156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.533+0000 7fe73be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73c0adcf0 0x7fe73c156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58408/0 (socket says 172.21.15.16:58408) 2024-08-17T02:45:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.533+0000 7fe73be00640 1 -- 172.21.15.16:0/4132422377 learned_addr learned my addr 172.21.15.16:0/4132422377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:32.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.534+0000 7fe733e00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe73c0b0a80 0x7fe73c16a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.534+0000 7fe740e00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe73c157e10 0x7fe73c16aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.534+0000 7fe73be00640 1 -- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe73c0b0a80 msgr2=0x7fe73c16a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.534+0000 7fe73be00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe73c0b0a80 0x7fe73c16a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.534+0000 7fe73be00640 1 -- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe73c157e10 msgr2=0x7fe73c16aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.534+0000 7fe73be00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe73c157e10 0x7fe73c16aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.534+0000 7fe73be00640 1 -- 172.21.15.16:0/4132422377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe724009390 con 0x7fe73c0adcf0 2024-08-17T02:45:32.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.534+0000 7fe73be00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73c0adcf0 0x7fe73c156420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0098b0 tx=0x7fe72c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:32.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.534+0000 7fe733400640 1 -- 172.21.15.16:0/4132422377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c03b610 con 0x7fe73c0adcf0 2024-08-17T02:45:32.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.535+0000 7fe733400640 1 -- 172.21.15.16:0/4132422377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72c03b770 con 0x7fe73c0adcf0 2024-08-17T02:45:32.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.535+0000 7fe733400640 1 -- 172.21.15.16:0/4132422377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72c03bb10 con 0x7fe73c0adcf0 2024-08-17T02:45:32.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.535+0000 7fe742e00640 1 -- 172.21.15.16:0/4132422377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72c009390 con 0x7fe73c0adcf0 2024-08-17T02:45:32.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.535+0000 7fe742e00640 1 -- 172.21.15.16:0/4132422377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73c13f110 con 0x7fe73c0adcf0 2024-08-17T02:45:32.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.536+0000 7fe742e00640 1 -- 172.21.15.16:0/4132422377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73c064db0 con 0x7fe73c0adcf0 2024-08-17T02:45:32.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.537+0000 7fe733400640 1 -- 172.21.15.16:0/4132422377 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fe72c043030 con 0x7fe73c0adcf0 2024-08-17T02:45:32.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.537+0000 7fe733400640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe718065180 0x7fe718067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.538+0000 7fe733400640 1 -- 172.21.15.16:0/4132422377 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe72c041070 con 0x7fe73c0adcf0 2024-08-17T02:45:32.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.538+0000 7fe733e00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe718065180 0x7fe718067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.539+0000 7fe733e00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe718065180 0x7fe718067630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe73c159650 tx=0x7fe7240090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:32.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.542+0000 7fe733400640 1 -- 172.21.15.16:0/4132422377 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe72c0709d0 con 0x7fe73c0adcf0 2024-08-17T02:45:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.728+0000 7fe742e00640 1 -- 172.21.15.16:0/4132422377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe73c1514c0 con 0x7fe73c0adcf0 2024-08-17T02:45:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.728+0000 7fe733400640 1 -- 172.21.15.16:0/4132422377 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe72c04b0a0 con 0x7fe73c0adcf0 2024-08-17T02:45:32.729 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-17T02:45:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.730+0000 7fe742e00640 1 -- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe718065180 msgr2=0x7fe718067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.730+0000 7fe742e00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe718065180 0x7fe718067630 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe73c159650 tx=0x7fe7240090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.730+0000 7fe742e00640 1 -- 172.21.15.16:0/4132422377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73c0adcf0 msgr2=0x7fe73c156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.730+0000 7fe742e00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73c0adcf0 0x7fe73c156420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0098b0 tx=0x7fe72c032000 comp rx=0 tx=0).stop 2024-08-17T02:45:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.730+0000 7fe742e00640 1 -- 172.21.15.16:0/4132422377 shutdown_connections 2024-08-17T02:45:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.730+0000 7fe742e00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe718065180 0x7fe718067630 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.730+0000 7fe742e00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe73c157e10 0x7fe73c16aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.730+0000 7fe742e00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe73c0b0a80 0x7fe73c16a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.730+0000 7fe742e00640 1 --2- 172.21.15.16:0/4132422377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe73c0adcf0 0x7fe73c156420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.730+0000 7fe742e00640 1 -- 172.21.15.16:0/4132422377 >> 172.21.15.16:0/4132422377 conn(0x7fe73c067750 msgr2=0x7fe73c14d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.730+0000 7fe742e00640 1 -- 172.21.15.16:0/4132422377 shutdown_connections 2024-08-17T02:45:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.731+0000 7fe742e00640 1 -- 172.21.15.16:0/4132422377 wait complete. 2024-08-17T02:45:32.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:45:32.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.898+0000 7f1e37200640 1 Processor -- start 2024-08-17T02:45:32.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.899+0000 7f1e37200640 1 -- start start 2024-08-17T02:45:32.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.900+0000 7f1e37200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1e380af0f0 0x7f1e380abc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.900+0000 7f1e37200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e380af520 0x7f1e380ac160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.900+0000 7f1e37200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e380ab790 con 0x7f1e380ae540 2024-08-17T02:45:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.900+0000 7f1e37200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e380aab80 con 0x7f1e380af520 2024-08-17T02:45:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.900+0000 7f1e37200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e380aae60 con 0x7f1e380af0f0 2024-08-17T02:45:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.900+0000 7f1e36800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e380af520 0x7f1e380ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.900+0000 7f1e36800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e380af520 0x7f1e380ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58416/0 (socket says 172.21.15.16:58416) 2024-08-17T02:45:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.900+0000 7f1e36800640 1 -- 172.21.15.16:0/1721633662 learned_addr learned my addr 172.21.15.16:0/1721633662 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.900+0000 7f1e35400640 1 --2- 172.21.15.16:0/1721633662 >> v2:172.21.15.142:3300/0 conn(0x7f1e380af0f0 0x7f1e380abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.900+0000 7f1e36800640 1 -- 172.21.15.16:0/1721633662 >> v2:172.21.15.142:3300/0 conn(0x7f1e380af0f0 msgr2=0x7f1e380abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.900+0000 7f1e36800640 1 --2- 172.21.15.16:0/1721633662 >> v2:172.21.15.142:3300/0 conn(0x7f1e380af0f0 0x7f1e380abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.900+0000 7f1e36800640 1 -- 172.21.15.16:0/1721633662 >> v1:172.21.15.142:6789/0 conn(0x7f1e380ae540 legacy=0x7f1e380ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.901+0000 7f1e36800640 1 -- 172.21.15.16:0/1721633662 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e380ab140 con 0x7f1e380af520 2024-08-17T02:45:32.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.901+0000 7f1e36800640 1 --2- 172.21.15.16:0/1721633662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e380af520 0x7f1e380ac160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1e380ae230 tx=0x7f1e30009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5fea2fbe910214 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.902+0000 7f1e34a00640 1 -- 172.21.15.16:0/1721633662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3002e050 con 0x7f1e380af520 2024-08-17T02:45:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.902+0000 7f1e34a00640 1 -- 172.21.15.16:0/1721633662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e30007400 con 0x7f1e380af520 2024-08-17T02:45:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.902+0000 7f1e34a00640 1 -- 172.21.15.16:0/1721633662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e300077e0 con 0x7f1e380af520 2024-08-17T02:45:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.902+0000 7f1e37200640 1 -- 172.21.15.16:0/1721633662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e380af520 msgr2=0x7f1e380ac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.902+0000 7f1e37200640 1 --2- 172.21.15.16:0/1721633662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e380af520 0x7f1e380ac160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1e380ae230 tx=0x7f1e30009840 comp rx=0 tx=0).stop 2024-08-17T02:45:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.902+0000 7f1e37200640 1 -- 172.21.15.16:0/1721633662 shutdown_connections 2024-08-17T02:45:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.902+0000 7f1e37200640 1 --2- 172.21.15.16:0/1721633662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e380af520 0x7f1e380ac160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.902+0000 7f1e37200640 1 --2- 172.21.15.16:0/1721633662 >> v2:172.21.15.142:3300/0 conn(0x7f1e380af0f0 0x7f1e380abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.902+0000 7f1e37200640 1 -- 172.21.15.16:0/1721633662 >> 172.21.15.16:0/1721633662 conn(0x7f1e38096a70 msgr2=0x7f1e38096e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.902+0000 7f1e37200640 1 -- 172.21.15.16:0/1721633662 shutdown_connections 2024-08-17T02:45:32.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.903+0000 7f1e37200640 1 -- 172.21.15.16:0/1721633662 wait complete. 2024-08-17T02:45:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.904+0000 7f1e37200640 1 Processor -- start 2024-08-17T02:45:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.904+0000 7f1e37200640 1 -- start start 2024-08-17T02:45:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e37200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e380ae540 0x7f1e3809aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e37200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e380af0f0 0x7f1e38155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e37200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e380af520 0x7f1e380b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e37200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e381561f0 con 0x7f1e380af520 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e37200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e38155f10 con 0x7f1e380ae540 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e37200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3815b620 con 0x7f1e380af0f0 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e36800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e380af520 0x7f1e380b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e36800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e380af520 0x7f1e380b1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58426/0 (socket says 172.21.15.16:58426) 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e36800640 1 -- 172.21.15.16:0/1311058024 learned_addr learned my addr 172.21.15.16:0/1311058024 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e35e00640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e380ae540 0x7f1e3809aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e35400640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e380af0f0 0x7f1e38155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e36800640 1 -- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e380af0f0 msgr2=0x7f1e38155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e36800640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e380af0f0 0x7f1e38155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e36800640 1 -- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e380ae540 msgr2=0x7f1e3809aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e36800640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e380ae540 0x7f1e3809aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.905+0000 7f1e36800640 1 -- 172.21.15.16:0/1311058024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e30009ae0 con 0x7f1e380af520 2024-08-17T02:45:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.906+0000 7f1e35e00640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e380ae540 0x7f1e3809aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.906+0000 7f1e35400640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e380af0f0 0x7f1e38155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.906+0000 7f1e36800640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e380af520 0x7f1e380b1770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009f50 tx=0x7f1e30037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.906+0000 7f1e2d600640 1 -- 172.21.15.16:0/1311058024 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30033e40 con 0x7f1e380af520 2024-08-17T02:45:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.906+0000 7f1e2d600640 1 -- 172.21.15.16:0/1311058024 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3003c410 con 0x7f1e380af520 2024-08-17T02:45:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.906+0000 7f1e2d600640 1 -- 172.21.15.16:0/1311058024 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3003c7d0 con 0x7f1e380af520 2024-08-17T02:45:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.906+0000 7f1e37200640 1 -- 172.21.15.16:0/1311058024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3815b900 con 0x7f1e380af520 2024-08-17T02:45:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.906+0000 7f1e37200640 1 -- 172.21.15.16:0/1311058024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3809b430 con 0x7f1e380af520 2024-08-17T02:45:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.907+0000 7f1e37200640 1 -- 172.21.15.16:0/1311058024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3815b480 con 0x7f1e380af520 2024-08-17T02:45:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.908+0000 7f1e2d600640 1 -- 172.21.15.16:0/1311058024 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1e30039050 con 0x7f1e380af520 2024-08-17T02:45:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.909+0000 7f1e2d600640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1e18065150 0x7f1e18067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.909+0000 7f1e2d600640 1 -- 172.21.15.16:0/1311058024 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e300a2fe0 con 0x7f1e380af520 2024-08-17T02:45:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.910+0000 7f1e35e00640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1e18065150 0x7f1e18067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.910+0000 7f1e35e00640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1e18065150 0x7f1e18067600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e28001820 tx=0x7f1e28009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:32.913+0000 7f1e2d600640 1 -- 172.21.15.16:0/1311058024 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1e3815b480 con 0x7f1e380af520 2024-08-17T02:45:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.093+0000 7f1e37200640 1 -- 172.21.15.16:0/1311058024 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e3815b480 con 0x7f1e380af520 2024-08-17T02:45:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.094+0000 7f1e2d600640 1 -- 172.21.15.16:0/1311058024 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e3815b480 con 0x7f1e380af520 2024-08-17T02:45:33.096 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:45:33.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.096+0000 7f1e37200640 1 -- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1e18065150 msgr2=0x7f1e18067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.096+0000 7f1e37200640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1e18065150 0x7f1e18067600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e28001820 tx=0x7f1e28009040 comp rx=0 tx=0).stop 2024-08-17T02:45:33.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.096+0000 7f1e37200640 1 -- 172.21.15.16:0/1311058024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e380af520 msgr2=0x7f1e380b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.096+0000 7f1e37200640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e380af520 0x7f1e380b1770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1e30009f50 tx=0x7f1e30037000 comp rx=0 tx=0).stop 2024-08-17T02:45:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.096+0000 7f1e37200640 1 -- 172.21.15.16:0/1311058024 shutdown_connections 2024-08-17T02:45:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.096+0000 7f1e37200640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1e18065150 0x7f1e18067600 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.096+0000 7f1e37200640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e380af520 0x7f1e380b1770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.096+0000 7f1e37200640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1e380af0f0 0x7f1e38155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.096+0000 7f1e37200640 1 --2- 172.21.15.16:0/1311058024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1e380ae540 0x7f1e3809aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.096+0000 7f1e37200640 1 -- 172.21.15.16:0/1311058024 >> 172.21.15.16:0/1311058024 conn(0x7f1e38096a70 msgr2=0x7f1e38068d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.096+0000 7f1e37200640 1 -- 172.21.15.16:0/1311058024 shutdown_connections 2024-08-17T02:45:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.096+0000 7f1e37200640 1 -- 172.21.15.16:0/1311058024 wait complete. 2024-08-17T02:45:33.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:45:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.264+0000 7f7aeb200640 1 Processor -- start 2024-08-17T02:45:33.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.265+0000 7f7aeb200640 1 -- start start 2024-08-17T02:45:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.266+0000 7f7aeb200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ae4159aa0 0x7f7ae40ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.266+0000 7f7aeb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ae40ccf00 0x7f7ae41629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.266+0000 7f7aeb200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae40cad60 con 0x7f7ae4159e70 2024-08-17T02:45:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.266+0000 7f7aeb200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae40c4b90 con 0x7f7ae4159aa0 2024-08-17T02:45:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.266+0000 7f7aeb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40ce630 con 0x7f7ae40ccf00 2024-08-17T02:45:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.266+0000 7f7ae9e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ae4159aa0 0x7f7ae40ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.266+0000 7f7ae9e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ae4159aa0 0x7f7ae40ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58432/0 (socket says 172.21.15.16:58432) 2024-08-17T02:45:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.266+0000 7f7ae9e00640 1 -- 172.21.15.16:0/3374931463 learned_addr learned my addr 172.21.15.16:0/3374931463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.266+0000 7f7aea800640 1 --2- 172.21.15.16:0/3374931463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ae40ccf00 0x7f7ae41629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.267+0000 7f7ae9e00640 1 -- 172.21.15.16:0/3374931463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ae40ccf00 msgr2=0x7f7ae41629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.267+0000 7f7ae9e00640 1 --2- 172.21.15.16:0/3374931463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ae40ccf00 0x7f7ae41629f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.267+0000 7f7ae9e00640 1 -- 172.21.15.16:0/3374931463 >> v1:172.21.15.142:6789/0 conn(0x7f7ae4159e70 legacy=0x7f7ae40cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.267+0000 7f7ae9e00640 1 -- 172.21.15.16:0/3374931463 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae40ce910 con 0x7f7ae4159aa0 2024-08-17T02:45:33.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.267+0000 7f7aea800640 1 --2- 172.21.15.16:0/3374931463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ae40ccf00 0x7f7ae41629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.268+0000 7f7ae9e00640 1 --2- 172.21.15.16:0/3374931463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ae4159aa0 0x7f7ae40ce050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009910 tx=0x7f7ad80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ed132175536843 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.268+0000 7f7ae8a00640 1 -- 172.21.15.16:0/3374931463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad802ae40 con 0x7f7ae4159aa0 2024-08-17T02:45:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.268+0000 7f7ae8a00640 1 -- 172.21.15.16:0/3374931463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad8032440 con 0x7f7ae4159aa0 2024-08-17T02:45:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.268+0000 7f7ae8a00640 1 -- 172.21.15.16:0/3374931463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8032820 con 0x7f7ae4159aa0 2024-08-17T02:45:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.268+0000 7f7aeb200640 1 -- 172.21.15.16:0/3374931463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ae4159aa0 msgr2=0x7f7ae40ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.268+0000 7f7aeb200640 1 --2- 172.21.15.16:0/3374931463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ae4159aa0 0x7f7ae40ce050 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009910 tx=0x7f7ad80091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.268+0000 7f7aeb200640 1 -- 172.21.15.16:0/3374931463 shutdown_connections 2024-08-17T02:45:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.268+0000 7f7aeb200640 1 --2- 172.21.15.16:0/3374931463 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ae40ccf00 0x7f7ae41629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.268+0000 7f7aeb200640 1 --2- 172.21.15.16:0/3374931463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ae4159aa0 0x7f7ae40ce050 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.268+0000 7f7aeb200640 1 -- 172.21.15.16:0/3374931463 >> 172.21.15.16:0/3374931463 conn(0x7f7ae4099040 msgr2=0x7f7ae406eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:33.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.269+0000 7f7aeb200640 1 -- 172.21.15.16:0/3374931463 shutdown_connections 2024-08-17T02:45:33.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.269+0000 7f7aeb200640 1 -- 172.21.15.16:0/3374931463 wait complete. 2024-08-17T02:45:33.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.270+0000 7f7aeb200640 1 Processor -- start 2024-08-17T02:45:33.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.270+0000 7f7aeb200640 1 -- start start 2024-08-17T02:45:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.271+0000 7f7aeb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae40ccf00 0x7f7ae40b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.271+0000 7f7aeb200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ae4159aa0 0x7f7ae40c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.271+0000 7f7aeb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ae4159e70 0x7f7ae416c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.271+0000 7f7aeb200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40b30d0 con 0x7f7ae4159aa0 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.271+0000 7f7aeb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40b33a0 con 0x7f7ae40ccf00 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.271+0000 7f7aeb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40b3680 con 0x7f7ae4159e70 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.271+0000 7f7ae9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ae4159aa0 0x7f7ae40c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.271+0000 7f7ae9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ae4159aa0 0x7f7ae40c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58434/0 (socket says 172.21.15.16:58434) 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.271+0000 7f7ae9400640 1 -- 172.21.15.16:0/4149767619 learned_addr learned my addr 172.21.15.16:0/4149767619 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.271+0000 7f7ae9e00640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae40ccf00 0x7f7ae40b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.271+0000 7f7aea800640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ae4159e70 0x7f7ae416c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.272+0000 7f7ae9400640 1 -- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ae4159e70 msgr2=0x7f7ae416c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.272+0000 7f7ae9400640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ae4159e70 0x7f7ae416c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.272+0000 7f7ae9400640 1 -- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae40ccf00 msgr2=0x7f7ae40b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.272+0000 7f7ae9400640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae40ccf00 0x7f7ae40b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.272+0000 7f7ae9400640 1 -- 172.21.15.16:0/4149767619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad8009450 con 0x7f7ae4159aa0 2024-08-17T02:45:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.272+0000 7f7ae9400640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ae4159aa0 0x7f7ae40c87d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400c4c0 tx=0x7f7ad4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:33.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.272+0000 7f7add600640 1 -- 172.21.15.16:0/4149767619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4011590 con 0x7f7ae4159aa0 2024-08-17T02:45:33.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.272+0000 7f7add600640 1 -- 172.21.15.16:0/4149767619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad4011710 con 0x7f7ae4159aa0 2024-08-17T02:45:33.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.272+0000 7f7add600640 1 -- 172.21.15.16:0/4149767619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4011ad0 con 0x7f7ae4159aa0 2024-08-17T02:45:33.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.272+0000 7f7aeb200640 1 -- 172.21.15.16:0/4149767619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae40b3960 con 0x7f7ae4159aa0 2024-08-17T02:45:33.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.272+0000 7f7aeb200640 1 -- 172.21.15.16:0/4149767619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae40b3f20 con 0x7f7ae4159aa0 2024-08-17T02:45:33.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.273+0000 7f7aeb200640 1 -- 172.21.15.16:0/4149767619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae40b4130 con 0x7f7ae4159aa0 2024-08-17T02:45:33.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.274+0000 7f7add600640 1 -- 172.21.15.16:0/4149767619 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f7ad401a360 con 0x7f7ae4159aa0 2024-08-17T02:45:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.275+0000 7f7add600640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7ac8065180 0x7f7ac8067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.275+0000 7f7add600640 1 -- 172.21.15.16:0/4149767619 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ad40814f0 con 0x7f7ae4159aa0 2024-08-17T02:45:33.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.276+0000 7f7ae9e00640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7ac8065180 0x7f7ac8067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:33.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.276+0000 7f7ae9e00640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7ac8065180 0x7f7ac8067630 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8034000 tx=0x7f7ad8032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:33.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.279+0000 7f7add600640 1 -- 172.21.15.16:0/4149767619 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7ad404f510 con 0x7f7ae4159aa0 2024-08-17T02:45:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.460+0000 7f7aeb200640 1 -- 172.21.15.16:0/4149767619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ae4071050 con 0x7f7ae4159aa0 2024-08-17T02:45:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.460+0000 7f7add600640 1 -- 172.21.15.16:0/4149767619 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ad4052b30 con 0x7f7ae4159aa0 2024-08-17T02:45:33.462 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-08-17T02:45:33.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.462+0000 7f7aeb200640 1 -- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7ac8065180 msgr2=0x7f7ac8067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.462+0000 7f7aeb200640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7ac8065180 0x7f7ac8067630 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8034000 tx=0x7f7ad8032000 comp rx=0 tx=0).stop 2024-08-17T02:45:33.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.462+0000 7f7aeb200640 1 -- 172.21.15.16:0/4149767619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ae4159aa0 msgr2=0x7f7ae40c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.462+0000 7f7aeb200640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ae4159aa0 0x7f7ae40c87d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400c4c0 tx=0x7f7ad4007ef0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.462+0000 7f7aeb200640 1 -- 172.21.15.16:0/4149767619 shutdown_connections 2024-08-17T02:45:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.462+0000 7f7aeb200640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7ac8065180 0x7f7ac8067630 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.462+0000 7f7aeb200640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7ae4159e70 0x7f7ae416c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.462+0000 7f7aeb200640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ae4159aa0 0x7f7ae40c87d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.462+0000 7f7aeb200640 1 --2- 172.21.15.16:0/4149767619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ae40ccf00 0x7f7ae40b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.462+0000 7f7aeb200640 1 -- 172.21.15.16:0/4149767619 >> 172.21.15.16:0/4149767619 conn(0x7f7ae4099040 msgr2=0x7f7ae40cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.462+0000 7f7aeb200640 1 -- 172.21.15.16:0/4149767619 shutdown_connections 2024-08-17T02:45:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.462+0000 7f7aeb200640 1 -- 172.21.15.16:0/4149767619 wait complete. 2024-08-17T02:45:33.478 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:45:33.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.627+0000 7f9115000640 1 Processor -- start 2024-08-17T02:45:33.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.627+0000 7f9115000640 1 -- start start 2024-08-17T02:45:33.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.628+0000 7f9115000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91100cf2d0 0x7f91100ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:33.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.628+0000 7f9115000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f91100ad6e0 0x7f91100adb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:33.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.628+0000 7f9115000640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91100cb730 con 0x7f91100ca350 2024-08-17T02:45:33.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.628+0000 7f9115000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91100cae30 con 0x7f91100ad6e0 2024-08-17T02:45:33.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.628+0000 7f9115000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91100cf9c0 con 0x7f91100cf2d0 2024-08-17T02:45:33.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.628+0000 7f910f400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f91100ca350 0x7f91100ca720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46254/0 (socket says 172.21.15.16:46254) 2024-08-17T02:45:33.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.628+0000 7f910f400640 1 -- 172.21.15.16:0/727210539 learned_addr learned my addr 172.21.15.16:0/727210539 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:33.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.628+0000 7f9107e00640 1 --2- 172.21.15.16:0/727210539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91100cf2d0 0x7f91100ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.628+0000 7f910fe00640 1 --2- 172.21.15.16:0/727210539 >> v2:172.21.15.142:3300/0 conn(0x7f91100ad6e0 0x7f91100adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.629+0000 7f910ea00640 1 -- 172.21.15.16:0/727210539 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511835221 0 0) 0x7f91100cb730 con 0x7f91100ca350 2024-08-17T02:45:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.629+0000 7f910ea00640 1 -- 172.21.15.16:0/727210539 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90f40034e0 con 0x7f91100ca350 2024-08-17T02:45:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.629+0000 7f9107e00640 1 -- 172.21.15.16:0/727210539 >> v2:172.21.15.142:3300/0 conn(0x7f91100ad6e0 msgr2=0x7f91100adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.629+0000 7f9107e00640 1 --2- 172.21.15.16:0/727210539 >> v2:172.21.15.142:3300/0 conn(0x7f91100ad6e0 0x7f91100adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.629+0000 7f9107e00640 1 -- 172.21.15.16:0/727210539 >> v1:172.21.15.16:6789/0 conn(0x7f91100ca350 legacy=0x7f91100ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.629+0000 7f9107e00640 1 -- 172.21.15.16:0/727210539 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100cb0e0 con 0x7f91100cf2d0 2024-08-17T02:45:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.630+0000 7f9107e00640 1 --2- 172.21.15.16:0/727210539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91100cf2d0 0x7f91100ad1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009850 tx=0x7f90f80090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e4ece72166ca557 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.630+0000 7f910ea00640 1 -- 172.21.15.16:0/727210539 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f802ae10 con 0x7f91100cf2d0 2024-08-17T02:45:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.630+0000 7f910ea00640 1 -- 172.21.15.16:0/727210539 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f8032440 con 0x7f91100cf2d0 2024-08-17T02:45:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.630+0000 7f910ea00640 1 -- 172.21.15.16:0/727210539 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f8032800 con 0x7f91100cf2d0 2024-08-17T02:45:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.631+0000 7f9115000640 1 -- 172.21.15.16:0/727210539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91100cf2d0 msgr2=0x7f91100ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.631+0000 7f9115000640 1 --2- 172.21.15.16:0/727210539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91100cf2d0 0x7f91100ad1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f90f8009850 tx=0x7f90f80090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.631+0000 7f9115000640 1 -- 172.21.15.16:0/727210539 shutdown_connections 2024-08-17T02:45:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.631+0000 7f9115000640 1 --2- 172.21.15.16:0/727210539 >> v2:172.21.15.142:3300/0 conn(0x7f91100ad6e0 0x7f91100adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.631+0000 7f9115000640 1 --2- 172.21.15.16:0/727210539 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91100cf2d0 0x7f91100ad1a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.631+0000 7f9115000640 1 -- 172.21.15.16:0/727210539 >> 172.21.15.16:0/727210539 conn(0x7f91100cfdb0 msgr2=0x7f91100d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.631+0000 7f9115000640 1 -- 172.21.15.16:0/727210539 shutdown_connections 2024-08-17T02:45:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.631+0000 7f9115000640 1 -- 172.21.15.16:0/727210539 wait complete. 2024-08-17T02:45:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.632+0000 7f9115000640 1 Processor -- start 2024-08-17T02:45:33.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.632+0000 7f9115000640 1 -- start start 2024-08-17T02:45:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.633+0000 7f9115000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100ad6e0 0x7f91100c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.633+0000 7f9115000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91100ca350 0x7f91100b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.633+0000 7f9115000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91100cf2d0 0x7f91100afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.633+0000 7f9115000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911009c470 con 0x7f91100ad6e0 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.633+0000 7f9115000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911009b5e0 con 0x7f91100ca350 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.633+0000 7f9115000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911009b860 con 0x7f91100cf2d0 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f910f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100ad6e0 0x7f91100c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f910f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100ad6e0 0x7f91100c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58442/0 (socket says 172.21.15.16:58442) 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f910f400640 1 -- 172.21.15.16:0/243756862 learned_addr learned my addr 172.21.15.16:0/243756862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f9107e00640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91100ca350 0x7f91100b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f910fe00640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91100cf2d0 0x7f91100afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f910f400640 1 -- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91100cf2d0 msgr2=0x7f91100afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f910f400640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91100cf2d0 0x7f91100afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f910f400640 1 -- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91100ca350 msgr2=0x7f91100b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f910f400640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91100ca350 0x7f91100b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f910f400640 1 -- 172.21.15.16:0/243756862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f8009390 con 0x7f91100ad6e0 2024-08-17T02:45:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f910fe00640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91100cf2d0 0x7f91100afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f9107e00640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91100ca350 0x7f91100b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f910f400640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100ad6e0 0x7f91100c56d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f910000c590 tx=0x7f9100007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.634+0000 7f9106a00640 1 -- 172.21.15.16:0/243756862 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100005aa0 con 0x7f91100ad6e0 2024-08-17T02:45:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.635+0000 7f9106a00640 1 -- 172.21.15.16:0/243756862 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9100005c00 con 0x7f91100ad6e0 2024-08-17T02:45:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.635+0000 7f9115000640 1 -- 172.21.15.16:0/243756862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911009bb40 con 0x7f91100ad6e0 2024-08-17T02:45:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.635+0000 7f9106a00640 1 -- 172.21.15.16:0/243756862 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000070f0 con 0x7f91100ad6e0 2024-08-17T02:45:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.635+0000 7f9115000640 1 -- 172.21.15.16:0/243756862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911009c100 con 0x7f91100ad6e0 2024-08-17T02:45:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.636+0000 7f9115000640 1 -- 172.21.15.16:0/243756862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9110088ed0 con 0x7f91100ad6e0 2024-08-17T02:45:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.637+0000 7f9106a00640 1 -- 172.21.15.16:0/243756862 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f91000074a0 con 0x7f91100ad6e0 2024-08-17T02:45:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.637+0000 7f9106a00640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f90ec065150 0x7f90ec067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.637+0000 7f9106a00640 1 -- 172.21.15.16:0/243756862 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9100081f10 con 0x7f91100ad6e0 2024-08-17T02:45:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.638+0000 7f9107e00640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f90ec065150 0x7f90ec067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:33.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.639+0000 7f9107e00640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f90ec065150 0x7f90ec067600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f90f8034000 tx=0x7f90f8032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:33.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.641+0000 7f9106a00640 1 -- 172.21.15.16:0/243756862 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9110088ed0 con 0x7f91100ad6e0 2024-08-17T02:45:33.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.866+0000 7f9115000640 1 -- 172.21.15.16:0/243756862 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9110088ed0 con 0x7f91100ad6e0 2024-08-17T02:45:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.866+0000 7f9106a00640 1 -- 172.21.15.16:0/243756862 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f9110088ed0 con 0x7f91100ad6e0 2024-08-17T02:45:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:33.868 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/111049205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":111049205},{"type":"v1","addr":"172.21.15.142:6836","nonce":111049205}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2692359383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2692359383},{"type":"v1","addr":"172.21.15.142:6837","nonce":2692359383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3179355872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3179355872},{"type":"v1","addr":"172.21.15.16:6833","nonce":3179355872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:45:19.450658+0000","modified":"2024-08-17T02:45:23.039406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2140506421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2140506421},{"type":"v1","addr":"172.21.15.16:6835","nonce":2140506421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:45:33.868 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-08-17T02:45:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.868+0000 7f9115000640 1 -- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f90ec065150 msgr2=0x7f90ec067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.868+0000 7f9115000640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f90ec065150 0x7f90ec067600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f90f8034000 tx=0x7f90f8032000 comp rx=0 tx=0).stop 2024-08-17T02:45:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.868+0000 7f9115000640 1 -- 172.21.15.16:0/243756862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100ad6e0 msgr2=0x7f91100c56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.868+0000 7f9115000640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100ad6e0 0x7f91100c56d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f910000c590 tx=0x7f9100007d60 comp rx=0 tx=0).stop 2024-08-17T02:45:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.869+0000 7f9115000640 1 -- 172.21.15.16:0/243756862 shutdown_connections 2024-08-17T02:45:33.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.869+0000 7f9115000640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f90ec065150 0x7f90ec067600 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.869+0000 7f9115000640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f91100cf2d0 0x7f91100afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.869+0000 7f9115000640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91100ca350 0x7f91100b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.869+0000 7f9115000640 1 --2- 172.21.15.16:0/243756862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91100ad6e0 0x7f91100c56d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.869+0000 7f9115000640 1 -- 172.21.15.16:0/243756862 >> 172.21.15.16:0/243756862 conn(0x7f91100cfdb0 msgr2=0x7f91100b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.869+0000 7f9115000640 1 -- 172.21.15.16:0/243756862 shutdown_connections 2024-08-17T02:45:33.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:33.869+0000 7f9115000640 1 -- 172.21.15.16:0/243756862 wait complete. 2024-08-17T02:45:33.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:34.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.034+0000 7f1d69000640 1 Processor -- start 2024-08-17T02:45:34.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.034+0000 7f1d69000640 1 -- start start 2024-08-17T02:45:34.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.035+0000 7f1d69000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1d64155a20 0x7f1d640adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:34.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.035+0000 7f1d69000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d64155df0 0x7f1d640ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.035+0000 7f1d69000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d64156670 con 0x7f1d6406d7f0 2024-08-17T02:45:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d69000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d640b0350 con 0x7f1d64155a20 2024-08-17T02:45:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d69000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6406e5e0 con 0x7f1d64155df0 2024-08-17T02:45:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d59400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d64155df0 0x7f1d640ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d61e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1d64155a20 0x7f1d640adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d59400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d64155df0 0x7f1d640ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:48396/0 (socket says 172.21.15.16:48396) 2024-08-17T02:45:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d62800640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f1d6406d7f0 0x7f1d640ac790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:45724/0 (socket says 172.21.15.16:45724) 2024-08-17T02:45:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d61e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1d64155a20 0x7f1d640adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:49974/0 (socket says 172.21.15.16:49974) 2024-08-17T02:45:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d59400640 1 -- 172.21.15.16:0/4215636292 learned_addr learned my addr 172.21.15.16:0/4215636292 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d61e00640 1 -- 172.21.15.16:0/4215636292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d64155df0 msgr2=0x7f1d640ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d61e00640 1 --2- 172.21.15.16:0/4215636292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d64155df0 0x7f1d640ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d61e00640 1 -- 172.21.15.16:0/4215636292 >> v1:172.21.15.142:6789/0 conn(0x7f1d6406d7f0 legacy=0x7f1d640ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d61e00640 1 -- 172.21.15.16:0/4215636292 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6406e8c0 con 0x7f1d64155a20 2024-08-17T02:45:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d59400640 1 --2- 172.21.15.16:0/4215636292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d64155df0 0x7f1d640ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:34.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.036+0000 7f1d61400640 1 -- 172.21.15.16:0/4215636292 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709017755 0 0) 0x7f1d64156670 con 0x7f1d6406d7f0 2024-08-17T02:45:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.037+0000 7f1d61e00640 1 --2- 172.21.15.16:0/4215636292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d64155a20 0x7f1d640adf10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009850 tx=0x7f1d540090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=185918dca17be9d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:34.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.038+0000 7f1d61400640 1 -- 172.21.15.16:0/4215636292 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5402ad90 con 0x7f1d64155a20 2024-08-17T02:45:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.038+0000 7f1d61400640 1 -- 172.21.15.16:0/4215636292 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d54032440 con 0x7f1d64155a20 2024-08-17T02:45:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.038+0000 7f1d61400640 1 -- 172.21.15.16:0/4215636292 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d540327e0 con 0x7f1d64155a20 2024-08-17T02:45:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.038+0000 7f1d69000640 1 -- 172.21.15.16:0/4215636292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d64155a20 msgr2=0x7f1d640adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.038+0000 7f1d69000640 1 --2- 172.21.15.16:0/4215636292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d64155a20 0x7f1d640adf10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009850 tx=0x7f1d540090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.038+0000 7f1d69000640 1 -- 172.21.15.16:0/4215636292 shutdown_connections 2024-08-17T02:45:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.038+0000 7f1d69000640 1 --2- 172.21.15.16:0/4215636292 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d64155df0 0x7f1d640ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.038+0000 7f1d69000640 1 --2- 172.21.15.16:0/4215636292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d64155a20 0x7f1d640adf10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.038+0000 7f1d69000640 1 -- 172.21.15.16:0/4215636292 >> 172.21.15.16:0/4215636292 conn(0x7f1d640cb9b0 msgr2=0x7f1d640cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.038+0000 7f1d69000640 1 -- 172.21.15.16:0/4215636292 shutdown_connections 2024-08-17T02:45:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.038+0000 7f1d69000640 1 -- 172.21.15.16:0/4215636292 wait complete. 2024-08-17T02:45:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.040+0000 7f1d69000640 1 Processor -- start 2024-08-17T02:45:34.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.040+0000 7f1d69000640 1 -- start start 2024-08-17T02:45:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.041+0000 7f1d69000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d6406d7f0 0x7f1d64142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.041+0000 7f1d69000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d64155a20 0x7f1d64144730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.041+0000 7f1d69000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d64155df0 0x7f1d64144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.041+0000 7f1d69000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64168a70 con 0x7f1d64155a20 2024-08-17T02:45:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.041+0000 7f1d69000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64148070 con 0x7f1d64155df0 2024-08-17T02:45:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.041+0000 7f1d69000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64168420 con 0x7f1d6406d7f0 2024-08-17T02:45:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.041+0000 7f1d59400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d64155a20 0x7f1d64144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.041+0000 7f1d59400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d64155a20 0x7f1d64144730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41568/0 (socket says 172.21.15.16:41568) 2024-08-17T02:45:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.041+0000 7f1d59400640 1 -- 172.21.15.16:0/497812841 learned_addr learned my addr 172.21.15.16:0/497812841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.041+0000 7f1d62800640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d64155df0 0x7f1d64144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.041+0000 7f1d61e00640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d6406d7f0 0x7f1d64142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.042+0000 7f1d59400640 1 -- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d6406d7f0 msgr2=0x7f1d64142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.042+0000 7f1d59400640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d6406d7f0 0x7f1d64142270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.042+0000 7f1d59400640 1 -- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d64155df0 msgr2=0x7f1d64144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.042+0000 7f1d59400640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d64155df0 0x7f1d64144c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.042+0000 7f1d59400640 1 -- 172.21.15.16:0/497812841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c009450 con 0x7f1d64155a20 2024-08-17T02:45:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.042+0000 7f1d61e00640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d6406d7f0 0x7f1d64142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.042+0000 7f1d59400640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d64155a20 0x7f1d64144730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00ea30 tx=0x7f1d4c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.042+0000 7f1d5a000640 1 -- 172.21.15.16:0/497812841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c00ca10 con 0x7f1d64155a20 2024-08-17T02:45:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.042+0000 7f1d5a000640 1 -- 172.21.15.16:0/497812841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c00cb70 con 0x7f1d64155a20 2024-08-17T02:45:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.042+0000 7f1d69000640 1 -- 172.21.15.16:0/497812841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d54009390 con 0x7f1d64155a20 2024-08-17T02:45:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.042+0000 7f1d5a000640 1 -- 172.21.15.16:0/497812841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c013470 con 0x7f1d64155a20 2024-08-17T02:45:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.042+0000 7f1d69000640 1 -- 172.21.15.16:0/497812841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d641427b0 con 0x7f1d64155a20 2024-08-17T02:45:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.043+0000 7f1d69000640 1 -- 172.21.15.16:0/497812841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d64168280 con 0x7f1d64155a20 2024-08-17T02:45:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.044+0000 7f1d5a000640 1 -- 172.21.15.16:0/497812841 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1d4c013820 con 0x7f1d64155a20 2024-08-17T02:45:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.045+0000 7f1d5a000640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1d40065150 0x7f1d40067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.045+0000 7f1d5a000640 1 -- 172.21.15.16:0/497812841 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1d4c0826f0 con 0x7f1d64155a20 2024-08-17T02:45:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.045+0000 7f1d61e00640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1d40065150 0x7f1d40067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.046+0000 7f1d61e00640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1d40065150 0x7f1d40067600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1d54034000 tx=0x7f1d54032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:34.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.049+0000 7f1d5a000640 1 -- 172.21.15.16:0/497812841 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1d64168280 con 0x7f1d64155a20 2024-08-17T02:45:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.232+0000 7f1d69000640 1 -- 172.21.15.16:0/497812841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d64168280 con 0x7f1d64155a20 2024-08-17T02:45:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.233+0000 7f1d5a000640 1 -- 172.21.15.16:0/497812841 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f1d64168280 con 0x7f1d64155a20 2024-08-17T02:45:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:34.235 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.235+0000 7f1d69000640 1 -- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1d40065150 msgr2=0x7f1d40067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.235+0000 7f1d69000640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1d40065150 0x7f1d40067600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1d54034000 tx=0x7f1d54032000 comp rx=0 tx=0).stop 2024-08-17T02:45:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.236+0000 7f1d69000640 1 -- 172.21.15.16:0/497812841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d64155a20 msgr2=0x7f1d64144730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.236+0000 7f1d69000640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d64155a20 0x7f1d64144730 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00ea30 tx=0x7f1d4c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.236+0000 7f1d69000640 1 -- 172.21.15.16:0/497812841 shutdown_connections 2024-08-17T02:45:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.236+0000 7f1d69000640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1d40065150 0x7f1d40067600 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.236+0000 7f1d69000640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d64155df0 0x7f1d64144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.236+0000 7f1d69000640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d64155a20 0x7f1d64144730 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.236+0000 7f1d69000640 1 --2- 172.21.15.16:0/497812841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1d6406d7f0 0x7f1d64142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.236+0000 7f1d69000640 1 -- 172.21.15.16:0/497812841 >> 172.21.15.16:0/497812841 conn(0x7f1d640cb9b0 msgr2=0x7f1d64158090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.236+0000 7f1d69000640 1 -- 172.21.15.16:0/497812841 shutdown_connections 2024-08-17T02:45:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.236+0000 7f1d69000640 1 -- 172.21.15.16:0/497812841 wait complete. 2024-08-17T02:45:34.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:45:34.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.403+0000 7f6fb7200640 1 Processor -- start 2024-08-17T02:45:34.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.404+0000 7f6fb7200640 1 -- start start 2024-08-17T02:45:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.405+0000 7f6fb7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb80aed50 0x7f6fb80ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:34.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.405+0000 7f6fb7200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fb80c5c40 0x7f6fb80caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.405+0000 7f6fb7200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb80cc890 con 0x7f6fb80ce6d0 2024-08-17T02:45:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.405+0000 7f6fb7200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb80ceb20 con 0x7f6fb80c5c40 2024-08-17T02:45:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.405+0000 7f6fb7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80cb4c0 con 0x7f6fb80aed50 2024-08-17T02:45:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.405+0000 7f6fb6800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fb80c5c40 0x7f6fb80caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.405+0000 7f6fb6800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fb80c5c40 0x7f6fb80caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41576/0 (socket says 172.21.15.16:41576) 2024-08-17T02:45:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.405+0000 7f6fb6800640 1 -- 172.21.15.16:0/262985886 learned_addr learned my addr 172.21.15.16:0/262985886 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.405+0000 7f6fb5e00640 1 --2- 172.21.15.16:0/262985886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb80aed50 0x7f6fb80ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.405+0000 7f6fb6800640 1 -- 172.21.15.16:0/262985886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb80aed50 msgr2=0x7f6fb80ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.406+0000 7f6fb6800640 1 --2- 172.21.15.16:0/262985886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb80aed50 0x7f6fb80ce190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.406+0000 7f6fb6800640 1 -- 172.21.15.16:0/262985886 >> v1:172.21.15.142:6789/0 conn(0x7f6fb80ce6d0 legacy=0x7f6fb80ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:34.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.406+0000 7f6fb6800640 1 -- 172.21.15.16:0/262985886 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb80cb7a0 con 0x7f6fb80c5c40 2024-08-17T02:45:34.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.406+0000 7f6fb6800640 1 --2- 172.21.15.16:0/262985886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb80c5c40 0x7f6fb80caec0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009f20 tx=0x7f6fb0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f37a3b79f19725 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:34.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.407+0000 7f6fb4a00640 1 -- 172.21.15.16:0/262985886 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0031400 con 0x7f6fb80c5c40 2024-08-17T02:45:34.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.407+0000 7f6fb4a00640 1 -- 172.21.15.16:0/262985886 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb002d050 con 0x7f6fb80c5c40 2024-08-17T02:45:34.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.407+0000 7f6fb4a00640 1 -- 172.21.15.16:0/262985886 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00317d0 con 0x7f6fb80c5c40 2024-08-17T02:45:34.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.407+0000 7f6fb7200640 1 -- 172.21.15.16:0/262985886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb80c5c40 msgr2=0x7f6fb80caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:34.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.407+0000 7f6fb7200640 1 --2- 172.21.15.16:0/262985886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb80c5c40 0x7f6fb80caec0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0009f20 tx=0x7f6fb0009840 comp rx=0 tx=0).stop 2024-08-17T02:45:34.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.407+0000 7f6fb7200640 1 -- 172.21.15.16:0/262985886 shutdown_connections 2024-08-17T02:45:34.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.407+0000 7f6fb7200640 1 --2- 172.21.15.16:0/262985886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb80c5c40 0x7f6fb80caec0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.407+0000 7f6fb7200640 1 --2- 172.21.15.16:0/262985886 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb80aed50 0x7f6fb80ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.407+0000 7f6fb7200640 1 -- 172.21.15.16:0/262985886 >> 172.21.15.16:0/262985886 conn(0x7f6fb8098e30 msgr2=0x7f6fb806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.407+0000 7f6fb7200640 1 -- 172.21.15.16:0/262985886 shutdown_connections 2024-08-17T02:45:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.407+0000 7f6fb7200640 1 -- 172.21.15.16:0/262985886 wait complete. 2024-08-17T02:45:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.409+0000 7f6fb7200640 1 Processor -- start 2024-08-17T02:45:34.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.409+0000 7f6fb7200640 1 -- start start 2024-08-17T02:45:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb80aed50 0x7f6fb80b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb80c5c40 0x7f6fb80afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb80ce6d0 0x7f6fb8148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80b01c0 con 0x7f6fb80aed50 2024-08-17T02:45:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80b2f50 con 0x7f6fb80c5c40 2024-08-17T02:45:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb80b3230 con 0x7f6fb80ce6d0 2024-08-17T02:45:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb80aed50 0x7f6fb80b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb80aed50 0x7f6fb80b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41586/0 (socket says 172.21.15.16:41586) 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb5e00640 1 -- 172.21.15.16:0/3511005412 learned_addr learned my addr 172.21.15.16:0/3511005412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb6800640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb80ce6d0 0x7f6fb8148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb5400640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb80c5c40 0x7f6fb80afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb5e00640 1 -- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb80ce6d0 msgr2=0x7f6fb8148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb5e00640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb80ce6d0 0x7f6fb8148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb5e00640 1 -- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb80c5c40 msgr2=0x7f6fb80afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb5e00640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb80c5c40 0x7f6fb80afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb5e00640 1 -- 172.21.15.16:0/3511005412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb0009ae0 con 0x7f6fb80aed50 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.410+0000 7f6fb5400640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb80c5c40 0x7f6fb80afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.411+0000 7f6fb5e00640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb80aed50 0x7f6fb80b2970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800d890 tx=0x7f6fa8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.411+0000 7f6fad600640 1 -- 172.21.15.16:0/3511005412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa800de20 con 0x7f6fb80aed50 2024-08-17T02:45:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.411+0000 7f6fad600640 1 -- 172.21.15.16:0/3511005412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa80098b0 con 0x7f6fb80aed50 2024-08-17T02:45:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.411+0000 7f6fad600640 1 -- 172.21.15.16:0/3511005412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8009c50 con 0x7f6fb80aed50 2024-08-17T02:45:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.411+0000 7f6fb7200640 1 -- 172.21.15.16:0/3511005412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb80b3510 con 0x7f6fb80aed50 2024-08-17T02:45:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.411+0000 7f6fb7200640 1 -- 172.21.15.16:0/3511005412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb80b3ad0 con 0x7f6fb80aed50 2024-08-17T02:45:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.412+0000 7f6fb7200640 1 -- 172.21.15.16:0/3511005412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb8071050 con 0x7f6fb80aed50 2024-08-17T02:45:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.413+0000 7f6fad600640 1 -- 172.21.15.16:0/3511005412 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f6fa8014620 con 0x7f6fb80aed50 2024-08-17T02:45:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.414+0000 7f6fad600640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6f98065150 0x7f6f98067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.414+0000 7f6fad600640 1 -- 172.21.15.16:0/3511005412 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6fa8082c90 con 0x7f6fb80aed50 2024-08-17T02:45:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.414+0000 7f6fb5400640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6f98065150 0x7f6f98067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.415+0000 7f6fb5400640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6f98065150 0x7f6f98067600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8155d20 tx=0x7f6fa4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.418+0000 7f6fad600640 1 -- 172.21.15.16:0/3511005412 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6fa8050cb0 con 0x7f6fb80aed50 2024-08-17T02:45:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.637+0000 7f6fb7200640 1 -- 172.21.15.16:0/3511005412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fb80ca040 con 0x7f6fb80aed50 2024-08-17T02:45:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.638+0000 7f6fad600640 1 -- 172.21.15.16:0/3511005412 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f6fa80542d0 con 0x7f6fb80aed50 2024-08-17T02:45:34.639 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:34.639 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/111049205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":111049205},{"type":"v1","addr":"172.21.15.142:6836","nonce":111049205}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2692359383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2692359383},{"type":"v1","addr":"172.21.15.142:6837","nonce":2692359383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3179355872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3179355872},{"type":"v1","addr":"172.21.15.16:6833","nonce":3179355872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:45:19.450658+0000","modified":"2024-08-17T02:45:23.039406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2140506421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2140506421},{"type":"v1","addr":"172.21.15.16:6835","nonce":2140506421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:45:34.640 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-08-17T02:45:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.640+0000 7f6fb7200640 1 -- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6f98065150 msgr2=0x7f6f98067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.640+0000 7f6fb7200640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6f98065150 0x7f6f98067600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8155d20 tx=0x7f6fa4009040 comp rx=0 tx=0).stop 2024-08-17T02:45:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.640+0000 7f6fb7200640 1 -- 172.21.15.16:0/3511005412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb80aed50 msgr2=0x7f6fb80b2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.640+0000 7f6fb7200640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb80aed50 0x7f6fb80b2970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800d890 tx=0x7f6fa8008040 comp rx=0 tx=0).stop 2024-08-17T02:45:34.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.640+0000 7f6fb7200640 1 -- 172.21.15.16:0/3511005412 shutdown_connections 2024-08-17T02:45:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.640+0000 7f6fb7200640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6f98065150 0x7f6f98067600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.640+0000 7f6fb7200640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6fb80ce6d0 0x7f6fb8148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.640+0000 7f6fb7200640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fb80c5c40 0x7f6fb80afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.640+0000 7f6fb7200640 1 --2- 172.21.15.16:0/3511005412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb80aed50 0x7f6fb80b2970 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.640+0000 7f6fb7200640 1 -- 172.21.15.16:0/3511005412 >> 172.21.15.16:0/3511005412 conn(0x7f6fb8098e30 msgr2=0x7f6fb8068890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.640+0000 7f6fb7200640 1 -- 172.21.15.16:0/3511005412 shutdown_connections 2024-08-17T02:45:34.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:34.640+0000 7f6fb7200640 1 -- 172.21.15.16:0/3511005412 wait complete. 2024-08-17T02:45:35.058 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-17T02:45:19.450658+0000', 'modified': '2024-08-17T02:45:23.039406+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4266}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4266': {'gid': 4266, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6835/2140506421', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6834', 'nonce': 2140506421}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 2140506421}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-08-17T02:45:35.059 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-17T02:45:35.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-08-17T02:45:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.212+0000 7fef51200640 1 Processor -- start 2024-08-17T02:45:35.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.212+0000 7fef51200640 1 -- start start 2024-08-17T02:45:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.213+0000 7fef51200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fef4c1401b0 0x7fef4c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.213+0000 7fef51200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef4c0b1440 0x7fef4c0c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.213+0000 7fef51200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef4c0caa80 con 0x7fef4c140580 2024-08-17T02:45:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.213+0000 7fef51200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c0cb450 con 0x7fef4c0b1440 2024-08-17T02:45:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.213+0000 7fef51200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c0c9e70 con 0x7fef4c1401b0 2024-08-17T02:45:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.214+0000 7fef4a800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef4c0b1440 0x7fef4c0c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.214+0000 7fef4a800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef4c0b1440 0x7fef4c0c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41600/0 (socket says 172.21.15.16:41600) 2024-08-17T02:45:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.214+0000 7fef4a800640 1 -- 172.21.15.16:0/4151060283 learned_addr learned my addr 172.21.15.16:0/4151060283 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.214+0000 7fef49e00640 1 --2- 172.21.15.16:0/4151060283 >> v2:172.21.15.142:3300/0 conn(0x7fef4c1401b0 0x7fef4c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.214+0000 7fef4a800640 1 -- 172.21.15.16:0/4151060283 >> v2:172.21.15.142:3300/0 conn(0x7fef4c1401b0 msgr2=0x7fef4c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.214+0000 7fef4a800640 1 --2- 172.21.15.16:0/4151060283 >> v2:172.21.15.142:3300/0 conn(0x7fef4c1401b0 0x7fef4c0caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.214+0000 7fef4a800640 1 -- 172.21.15.16:0/4151060283 >> v1:172.21.15.142:6789/0 conn(0x7fef4c140580 legacy=0x7fef4c0b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.214+0000 7fef4a800640 1 -- 172.21.15.16:0/4151060283 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c0ca150 con 0x7fef4c0b1440 2024-08-17T02:45:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.215+0000 7fef4a800640 1 --2- 172.21.15.16:0/4151060283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4c0b1440 0x7fef4c0c9040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef4c068650 tx=0x7fef44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b673246f28f05f12 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.215+0000 7fef48a00640 1 -- 172.21.15.16:0/4151060283 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef44031400 con 0x7fef4c0b1440 2024-08-17T02:45:35.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.215+0000 7fef48a00640 1 -- 172.21.15.16:0/4151060283 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4402d050 con 0x7fef4c0b1440 2024-08-17T02:45:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.215+0000 7fef48a00640 1 -- 172.21.15.16:0/4151060283 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef440317d0 con 0x7fef4c0b1440 2024-08-17T02:45:35.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.216+0000 7fef51200640 1 -- 172.21.15.16:0/4151060283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4c0b1440 msgr2=0x7fef4c0c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.216+0000 7fef51200640 1 --2- 172.21.15.16:0/4151060283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4c0b1440 0x7fef4c0c9040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef4c068650 tx=0x7fef44009840 comp rx=0 tx=0).stop 2024-08-17T02:45:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.216+0000 7fef51200640 1 -- 172.21.15.16:0/4151060283 shutdown_connections 2024-08-17T02:45:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.216+0000 7fef51200640 1 --2- 172.21.15.16:0/4151060283 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4c0b1440 0x7fef4c0c9040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.216+0000 7fef51200640 1 --2- 172.21.15.16:0/4151060283 >> v2:172.21.15.142:3300/0 conn(0x7fef4c1401b0 0x7fef4c0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.216+0000 7fef51200640 1 -- 172.21.15.16:0/4151060283 >> 172.21.15.16:0/4151060283 conn(0x7fef4c0cb9b0 msgr2=0x7fef4c0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.216+0000 7fef51200640 1 -- 172.21.15.16:0/4151060283 shutdown_connections 2024-08-17T02:45:35.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.216+0000 7fef51200640 1 -- 172.21.15.16:0/4151060283 wait complete. 2024-08-17T02:45:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.217+0000 7fef51200640 1 Processor -- start 2024-08-17T02:45:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.217+0000 7fef51200640 1 -- start start 2024-08-17T02:45:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.218+0000 7fef51200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4c0b1440 0x7fef4c13d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.218+0000 7fef51200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef4c1401b0 0x7fef4c0abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.218+0000 7fef51200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef4c140580 0x7fef4c069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.218+0000 7fef51200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0ad480 con 0x7fef4c0b1440 2024-08-17T02:45:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.218+0000 7fef51200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c13d6c0 con 0x7fef4c1401b0 2024-08-17T02:45:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.218+0000 7fef51200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0ac870 con 0x7fef4c140580 2024-08-17T02:45:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef49e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4c0b1440 0x7fef4c13d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef49e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4c0b1440 0x7fef4c13d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41612/0 (socket says 172.21.15.16:41612) 2024-08-17T02:45:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef49e00640 1 -- 172.21.15.16:0/3215933335 learned_addr learned my addr 172.21.15.16:0/3215933335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:35.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef49400640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef4c1401b0 0x7fef4c0abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef4a800640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef4c140580 0x7fef4c069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef49e00640 1 -- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef4c140580 msgr2=0x7fef4c069860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:35.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef49e00640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef4c140580 0x7fef4c069860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef49e00640 1 -- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef4c1401b0 msgr2=0x7fef4c0abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:35.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef49e00640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef4c1401b0 0x7fef4c0abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef49e00640 1 -- 172.21.15.16:0/3215933335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef44009ae0 con 0x7fef4c0b1440 2024-08-17T02:45:35.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef4a800640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef4c140580 0x7fef4c069860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:35.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef49400640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef4c1401b0 0x7fef4c0abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:35.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.219+0000 7fef49e00640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4c0b1440 0x7fef4c13d180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00d8c0 tx=0x7fef3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:35.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.220+0000 7fef41600640 1 -- 172.21.15.16:0/3215933335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00de50 con 0x7fef4c0b1440 2024-08-17T02:45:35.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.220+0000 7fef41600640 1 -- 172.21.15.16:0/3215933335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c00b8d0 con 0x7fef4c0b1440 2024-08-17T02:45:35.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.220+0000 7fef41600640 1 -- 172.21.15.16:0/3215933335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00bc70 con 0x7fef4c0b1440 2024-08-17T02:45:35.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.220+0000 7fef51200640 1 -- 172.21.15.16:0/3215933335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4c0acb50 con 0x7fef4c0b1440 2024-08-17T02:45:35.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.220+0000 7fef51200640 1 -- 172.21.15.16:0/3215933335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c0ad110 con 0x7fef4c0b1440 2024-08-17T02:45:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.222+0000 7fef41600640 1 -- 172.21.15.16:0/3215933335 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fef3c014640 con 0x7fef4c0b1440 2024-08-17T02:45:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.223+0000 7fef41600640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fef2c065150 0x7fef2c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.223+0000 7fef41600640 1 -- 172.21.15.16:0/3215933335 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef3c082a80 con 0x7fef4c0b1440 2024-08-17T02:45:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.223+0000 7fef49400640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fef2c065150 0x7fef2c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.223+0000 7fef5366e740 1 Processor -- start 2024-08-17T02:45:35.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.224+0000 7fef5366e740 1 -- start start 2024-08-17T02:45:35.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.224+0000 7fef49400640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fef2c065150 0x7fef2c067600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0a7090 tx=0x7fef38009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.224+0000 7fef5366e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba17a30 0x55c35ba17e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.224+0000 7fef5366e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35b9f30a0 0x55c35ba355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.224+0000 7fef5366e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35b9ec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.224+0000 7fef5366e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35b9ad380 con 0x55c35ba17a30 2024-08-17T02:45:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef5366e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35b9569e0 con 0x55c35b9f30a0 2024-08-17T02:45:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef5366e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c35b954170 con 0x55c35ba35b10 2024-08-17T02:45:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef49e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba17a30 0x55c35ba17e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef49e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba17a30 0x55c35ba17e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41614/0 (socket says 172.21.15.16:41614) 2024-08-17T02:45:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef49e00640 1 -- 172.21.15.16:0/3223813385 learned_addr learned my addr 172.21.15.16:0/3223813385 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef4a800640 1 --2- 172.21.15.16:0/3223813385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35b9ec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef49400640 1 --2- 172.21.15.16:0/3223813385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35b9f30a0 0x55c35ba355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef49e00640 1 -- 172.21.15.16:0/3223813385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 msgr2=0x55c35b9ec900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef49e00640 1 --2- 172.21.15.16:0/3223813385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35b9ec900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef49e00640 1 -- 172.21.15.16:0/3223813385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35b9f30a0 msgr2=0x55c35ba355d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef49e00640 1 --2- 172.21.15.16:0/3223813385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35b9f30a0 0x55c35ba355d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef49e00640 1 -- 172.21.15.16:0/3223813385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c35b903820 con 0x55c35ba17a30 2024-08-17T02:45:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef49400640 1 --2- 172.21.15.16:0/3223813385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35b9f30a0 0x55c35ba355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.225+0000 7fef49e00640 1 --2- 172.21.15.16:0/3223813385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba17a30 0x55c35ba17e00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00e010 tx=0x7fef3c080f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab65dd53abb671a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.226+0000 7fef36000640 1 -- 172.21.15.16:0/3223813385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c057a50 con 0x55c35ba17a30 2024-08-17T02:45:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.226+0000 7fef36000640 1 -- 172.21.15.16:0/3223813385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c05b7a0 con 0x55c35ba17a30 2024-08-17T02:45:35.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.226+0000 7fef36000640 1 -- 172.21.15.16:0/3223813385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c052940 con 0x55c35ba17a30 2024-08-17T02:45:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.226+0000 7fef5366e740 1 -- 172.21.15.16:0/3223813385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba17a30 msgr2=0x55c35ba17e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.226+0000 7fef5366e740 1 --2- 172.21.15.16:0/3223813385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba17a30 0x55c35ba17e00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00e010 tx=0x7fef3c080f00 comp rx=0 tx=0).stop 2024-08-17T02:45:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.226+0000 7fef5366e740 1 -- 172.21.15.16:0/3223813385 shutdown_connections 2024-08-17T02:45:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.226+0000 7fef5366e740 1 --2- 172.21.15.16:0/3223813385 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35b9ec900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.226+0000 7fef5366e740 1 --2- 172.21.15.16:0/3223813385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35b9f30a0 0x55c35ba355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.226+0000 7fef5366e740 1 --2- 172.21.15.16:0/3223813385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba17a30 0x55c35ba17e00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.226+0000 7fef5366e740 1 -- 172.21.15.16:0/3223813385 >> 172.21.15.16:0/3223813385 conn(0x55c35b98b030 msgr2=0x55c35ba39c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:35.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.226+0000 7fef5366e740 1 -- 172.21.15.16:0/3223813385 shutdown_connections 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.226+0000 7fef5366e740 1 -- 172.21.15.16:0/3223813385 wait complete. 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.227+0000 7fef5366e740 1 -- start start 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.227+0000 7fef5366e740 1 Processor -- start 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.228+0000 7fef5366e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35b9f30a0 0x55c35ba1d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.228+0000 7fef5366e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba13390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.228+0000 7fef5366e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba35b10 0x55c35ba382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.228+0000 7fef5366e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35b94ff60 con 0x55c35ba35b10 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.228+0000 7fef5366e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35b8f8690 con 0x55c35ba17a30 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.228+0000 7fef5366e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c35b8f8330 con 0x55c35b9f30a0 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.228+0000 7fef5366e740 10 client.0 _ll_register_callbacks cb 0x55c35b8f22b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.228+0000 7fef4a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba35b10 0x55c35ba382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.228+0000 7fef4a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba35b10 0x55c35ba382f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41620/0 (socket says 172.21.15.16:41620) 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.228+0000 7fef4a800640 1 -- 172.21.15.16:0/3436151168 learned_addr learned my addr 172.21.15.16:0/3436151168 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:35.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.228+0000 7fef49400640 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba13390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.228+0000 7fef49e00640 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35b9f30a0 0x55c35ba1d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef4a800640 1 -- 172.21.15.16:0/3436151168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35b9f30a0 msgr2=0x55c35ba1d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef4a800640 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35b9f30a0 0x55c35ba1d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef4a800640 1 -- 172.21.15.16:0/3436151168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 msgr2=0x55c35ba13390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef4a800640 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba13390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef4a800640 1 -- 172.21.15.16:0/3436151168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c0ad110 con 0x55c35ba35b10 2024-08-17T02:45:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef4a800640 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba35b10 0x55c35ba382f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef44009f50 tx=0x7fef44036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56781cee75b3a1a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-17T02:45:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef2aa00640 1 -- 172.21.15.16:0/3436151168 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4403dac0 con 0x55c35ba35b10 2024-08-17T02:45:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef2aa00640 1 -- 172.21.15.16:0/3436151168 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef4403dc20 con 0x55c35ba35b10 2024-08-17T02:45:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef5366e740 1 -- 172.21.15.16:0/3436151168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c35b983d20 con 0x55c35ba35b10 2024-08-17T02:45:35.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef2aa00640 1 -- 172.21.15.16:0/3436151168 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef440453f0 con 0x55c35ba35b10 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef5366e740 10 client.4644 fetch_fsmap learned FSMap version 8 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.229+0000 7fef5366e740 1 -- 172.21.15.16:0/3436151168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55c35b919ab0 con 0x55c35ba35b10 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.230+0000 7fef2aa00640 1 -- 172.21.15.16:0/3436151168 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c35b983d20 con 0x55c35ba35b10 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.230+0000 7fef2aa00640 1 -- 172.21.15.16:0/3436151168 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fef44045550 con 0x55c35ba35b10 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.230+0000 7fef5366e740 10 client.4644 fetch_fsmap finished waiting for FSMap version 8 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.230+0000 7fef5366e740 10 client.4644 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.230+0000 7fef5366e740 20 client.4644 populate_metadata read hostname 'smithi016' 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.230+0000 7fef5366e740 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.16:6834/2140506421,v1:172.21.15.16:6835/2140506421] conn(0x55c35ba36e40 0x55c35ba37210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.230+0000 7fef5366e740 4 client.4644 mds_command: new command op to 4266 tid=0[{"prefix": "get_command_descriptions"}] 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.230+0000 7fef5366e740 1 -- 172.21.15.16:0/3436151168 --> [v2:172.21.15.16:6834/2140506421,v1:172.21.15.16:6835/2140506421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c35b6573c0 con 0x55c35ba36e40 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.230+0000 7fef49e00640 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.16:6834/2140506421,v1:172.21.15.16:6835/2140506421] conn(0x55c35ba36e40 0x55c35ba37210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.231+0000 7fef49e00640 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.16:6834/2140506421,v1:172.21.15.16:6835/2140506421] conn(0x55c35ba36e40 0x55c35ba37210 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4da5e58056666ee4 server_cookie=35b99a010175c1b2 in_seq=0 out_seq=0 2024-08-17T02:45:35.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.231+0000 7fef2aa00640 10 client.4644 ms_handle_connect on v2:172.21.15.16:6834/2140506421 2024-08-17T02:45:35.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.231+0000 7fef2aa00640 1 -- 172.21.15.16:0/3436151168 <== mds.0 v2:172.21.15.16:6834/2140506421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (crc 0 0 0) 0x55c35b6573c0 con 0x55c35ba36e40 2024-08-17T02:45:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.231+0000 7fef2aa00640 10 client.4644 handle_command_reply: tid=0 2024-08-17T02:45:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.231+0000 7fef5366e740 1 client.4644 shutdown 2024-08-17T02:45:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.232+0000 7fef5366e740 1 -- 172.21.15.16:0/3436151168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba35b10 msgr2=0x55c35ba382f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.232+0000 7fef5366e740 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba35b10 0x55c35ba382f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fef44009f50 tx=0x7fef44036000 comp rx=0 tx=0).stop 2024-08-17T02:45:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.232+0000 7fef5366e740 1 -- 172.21.15.16:0/3436151168 shutdown_connections 2024-08-17T02:45:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.232+0000 7fef5366e740 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.16:6834/2140506421,v1:172.21.15.16:6835/2140506421] conn(0x55c35ba36e40 0x55c35ba37210 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.232+0000 7fef5366e740 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35ba35b10 0x55c35ba382f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.232+0000 7fef5366e740 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba13390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.232+0000 7fef5366e740 1 --2- 172.21.15.16:0/3436151168 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35b9f30a0 0x55c35ba1d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.232+0000 7fef2aa00640 0 client.4644 ms_handle_reset on v2:172.21.15.16:6834/2140506421 2024-08-17T02:45:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.232+0000 7fef5366e740 1 -- 172.21.15.16:0/3436151168 >> 172.21.15.16:0/3436151168 conn(0x55c35b98b030 msgr2=0x55c35ba16270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:35.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.232+0000 7fef5366e740 1 -- 172.21.15.16:0/3436151168 shutdown_connections 2024-08-17T02:45:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.232+0000 7fef5366e740 1 -- 172.21.15.16:0/3436151168 wait complete. 2024-08-17T02:45:35.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.232+0000 7fef5366e740 20 client.4644 trim_cache size 0 max 16384 2024-08-17T02:45:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.248+0000 7fef5366e740 1 Processor -- start 2024-08-17T02:45:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.248+0000 7fef5366e740 1 -- start start 2024-08-17T02:45:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef5366e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 0x55c35ba1cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef5366e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba37210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef5366e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35ba37750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef5366e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35b6caf20 con 0x55c35b9f30a0 2024-08-17T02:45:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef5366e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35b9e7760 con 0x55c35ba17a30 2024-08-17T02:45:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef5366e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c35ba17310 con 0x55c35ba35b10 2024-08-17T02:45:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef49e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 0x55c35ba1cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef49e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 0x55c35ba1cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41628/0 (socket says 172.21.15.16:41628) 2024-08-17T02:45:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef49e00640 1 -- 172.21.15.16:0/2878091702 learned_addr learned my addr 172.21.15.16:0/2878091702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef49400640 1 --2- 172.21.15.16:0/2878091702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba37210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef49e00640 1 -- 172.21.15.16:0/2878091702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 msgr2=0x55c35ba37750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef4a800640 1 --2- 172.21.15.16:0/2878091702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35ba37750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef49e00640 1 --2- 172.21.15.16:0/2878091702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35ba37750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef49e00640 1 -- 172.21.15.16:0/2878091702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 msgr2=0x55c35ba37210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef49e00640 1 --2- 172.21.15.16:0/2878091702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba37210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef49e00640 1 -- 172.21.15.16:0/2878091702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c35b9ed1a0 con 0x55c35b9f30a0 2024-08-17T02:45:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef49400640 1 --2- 172.21.15.16:0/2878091702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba37210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.249+0000 7fef4a800640 1 --2- 172.21.15.16:0/2878091702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35ba37750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef49e00640 1 --2- 172.21.15.16:0/2878091702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 0x55c35ba1cd00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fef3c050d00 tx=0x7fef3c061490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec29ccc327e0bdbf server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef36a00640 1 -- 172.21.15.16:0/2878091702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c014430 con 0x55c35b9f30a0 2024-08-17T02:45:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef36a00640 1 -- 172.21.15.16:0/2878091702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c00bdd0 con 0x55c35b9f30a0 2024-08-17T02:45:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef36a00640 1 -- 172.21.15.16:0/2878091702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c052940 con 0x55c35b9f30a0 2024-08-17T02:45:35.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef5366e740 1 -- 172.21.15.16:0/2878091702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 msgr2=0x55c35ba1cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:35.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef5366e740 1 --2- 172.21.15.16:0/2878091702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 0x55c35ba1cd00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fef3c050d00 tx=0x7fef3c061490 comp rx=0 tx=0).stop 2024-08-17T02:45:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef5366e740 1 -- 172.21.15.16:0/2878091702 shutdown_connections 2024-08-17T02:45:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef5366e740 1 --2- 172.21.15.16:0/2878091702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35ba37750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef5366e740 1 --2- 172.21.15.16:0/2878091702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba37210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef5366e740 1 --2- 172.21.15.16:0/2878091702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 0x55c35ba1cd00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef5366e740 1 -- 172.21.15.16:0/2878091702 >> 172.21.15.16:0/2878091702 conn(0x55c35b98b030 msgr2=0x55c35ba18e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef5366e740 1 -- 172.21.15.16:0/2878091702 shutdown_connections 2024-08-17T02:45:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.250+0000 7fef5366e740 1 -- 172.21.15.16:0/2878091702 wait complete. 2024-08-17T02:45:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.251+0000 7fef5366e740 1 -- start start 2024-08-17T02:45:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.251+0000 7fef5366e740 1 Processor -- start 2024-08-17T02:45:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.252+0000 7fef5366e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 0x55c35ba94d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.252+0000 7fef5366e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba55fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.252+0000 7fef5366e740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35ba11670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.252+0000 7fef5366e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35ba37e90 con 0x55c35b9f30a0 2024-08-17T02:45:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.252+0000 7fef5366e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35ba53ac0 con 0x55c35ba17a30 2024-08-17T02:45:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.252+0000 7fef5366e740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c35ba121d0 con 0x55c35ba35b10 2024-08-17T02:45:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.252+0000 7fef5366e740 10 client.0 _ll_register_callbacks cb 0x7fef440099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-17T02:45:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.252+0000 7fef49e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 0x55c35ba94d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.252+0000 7fef49e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 0x55c35ba94d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41630/0 (socket says 172.21.15.16:41630) 2024-08-17T02:45:35.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.252+0000 7fef49e00640 1 -- 172.21.15.16:0/2615822026 learned_addr learned my addr 172.21.15.16:0/2615822026 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.252+0000 7fef4a800640 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35ba11670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.252+0000 7fef49400640 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba55fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef49e00640 1 -- 172.21.15.16:0/2615822026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 msgr2=0x55c35ba11670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef49e00640 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35ba11670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef49e00640 1 -- 172.21.15.16:0/2615822026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 msgr2=0x55c35ba55fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef49e00640 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba55fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef49e00640 1 -- 172.21.15.16:0/2615822026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c35b903820 con 0x55c35b9f30a0 2024-08-17T02:45:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef49e00640 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 0x55c35ba94d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef3c054790 tx=0x7fef3c063040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773840dc3f75db2 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef2aa00640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-17T02:45:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef2aa00640 1 -- 172.21.15.16:0/2615822026 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00c150 con 0x55c35b9f30a0 2024-08-17T02:45:35.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef2aa00640 1 -- 172.21.15.16:0/2615822026 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef3c00c2b0 con 0x55c35b9f30a0 2024-08-17T02:45:35.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef5366e740 1 -- 172.21.15.16:0/2615822026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef3c05b7a0 con 0x55c35b9f30a0 2024-08-17T02:45:35.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef2aa00640 1 -- 172.21.15.16:0/2615822026 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef3c053cf0 con 0x55c35b9f30a0 2024-08-17T02:45:35.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef5366e740 10 client.4650 fetch_fsmap learned FSMap version 8 2024-08-17T02:45:35.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.253+0000 7fef5366e740 1 -- 172.21.15.16:0/2615822026 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55c35b91d490 con 0x55c35b9f30a0 2024-08-17T02:45:35.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.254+0000 7fef2aa00640 1 -- 172.21.15.16:0/2615822026 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c05b7a0 con 0x55c35b9f30a0 2024-08-17T02:45:35.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.254+0000 7fef2aa00640 1 -- 172.21.15.16:0/2615822026 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fef3c062050 con 0x55c35b9f30a0 2024-08-17T02:45:35.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.254+0000 7fef5366e740 10 client.4650 fetch_fsmap finished waiting for FSMap version 8 2024-08-17T02:45:35.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.254+0000 7fef5366e740 10 client.4650 resolve_mds: resolved name 'c' to daemon mds.c 2024-08-17T02:45:35.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.254+0000 7fef5366e740 20 client.4650 populate_metadata read hostname 'smithi016' 2024-08-17T02:45:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.254+0000 7fef5366e740 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.16:6834/2140506421,v1:172.21.15.16:6835/2140506421] conn(0x55c35ba50170 0x55c35ba1b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.254+0000 7fef5366e740 4 client.4650 mds_command: new command op to 4266 tid=0[{"prefix": "status"}] 2024-08-17T02:45:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.254+0000 7fef5366e740 1 -- 172.21.15.16:0/2615822026 --> [v2:172.21.15.16:6834/2140506421,v1:172.21.15.16:6835/2140506421] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c35b6686c0 con 0x55c35ba50170 2024-08-17T02:45:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.254+0000 7fef49400640 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.16:6834/2140506421,v1:172.21.15.16:6835/2140506421] conn(0x55c35ba50170 0x55c35ba1b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.254+0000 7fef2aa00640 10 client.4650 ms_handle_connect on v2:172.21.15.16:6834/2140506421 2024-08-17T02:45:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.254+0000 7fef49400640 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.16:6834/2140506421,v1:172.21.15.16:6835/2140506421] conn(0x55c35ba50170 0x55c35ba1b650 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bd1dc058e411a56 server_cookie=3dd32f18f0262af1 in_seq=0 out_seq=0 2024-08-17T02:45:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef2aa00640 1 -- 172.21.15.16:0/2615822026 <== mds.0 v2:172.21.15.16:6834/2140506421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55c35b6686c0 con 0x55c35ba50170 2024-08-17T02:45:35.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef2aa00640 10 client.4650 handle_command_reply: tid=0 2024-08-17T02:45:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef5366e740 1 client.4650 shutdown 2024-08-17T02:45:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef5366e740 1 -- 172.21.15.16:0/2615822026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 msgr2=0x55c35ba94d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef5366e740 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 0x55c35ba94d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef3c054790 tx=0x7fef3c063040 comp rx=0 tx=0).stop 2024-08-17T02:45:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef5366e740 1 -- 172.21.15.16:0/2615822026 shutdown_connections 2024-08-17T02:45:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef5366e740 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.16:6834/2140506421,v1:172.21.15.16:6835/2140506421] conn(0x55c35ba50170 0x55c35ba1b650 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef5366e740 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x55c35ba35b10 0x55c35ba11670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef5366e740 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c35ba17a30 0x55c35ba55fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef5366e740 1 --2- 172.21.15.16:0/2615822026 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c35b9f30a0 0x55c35ba94d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef2aa00640 0 client.4650 ms_handle_reset on v2:172.21.15.16:6834/2140506421 2024-08-17T02:45:35.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef5366e740 1 -- 172.21.15.16:0/2615822026 >> 172.21.15.16:0/2615822026 conn(0x55c35b98b030 msgr2=0x55c35ba38d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:35.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef5366e740 1 -- 172.21.15.16:0/2615822026 shutdown_connections 2024-08-17T02:45:35.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef5366e740 1 -- 172.21.15.16:0/2615822026 wait complete. 2024-08-17T02:45:35.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.255+0000 7fef5366e740 20 client.4650 trim_cache size 0 max 16384 2024-08-17T02:45:35.265 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-17T02:45:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778", 2024-08-17T02:45:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-08-17T02:45:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4266, 2024-08-17T02:45:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-08-17T02:45:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-08-17T02:45:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-08-17T02:45:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 15.228019895999999, 2024-08-17T02:45:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 8, 2024-08-17T02:45:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 21, 2024-08-17T02:45:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 21, 2024-08-17T02:45:35.266 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 21.942321321000001 2024-08-17T02:45:35.267 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-17T02:45:35.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.256+0000 7fef51200640 1 -- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fef2c065150 msgr2=0x7fef2c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:35.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.256+0000 7fef51200640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fef2c065150 0x7fef2c067600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0a7090 tx=0x7fef38009040 comp rx=0 tx=0).stop 2024-08-17T02:45:35.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.256+0000 7fef51200640 1 -- 172.21.15.16:0/3215933335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4c0b1440 msgr2=0x7fef4c13d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:35.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.256+0000 7fef51200640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4c0b1440 0x7fef4c13d180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00d8c0 tx=0x7fef3c008040 comp rx=0 tx=0).stop 2024-08-17T02:45:35.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.256+0000 7fef51200640 1 -- 172.21.15.16:0/3215933335 shutdown_connections 2024-08-17T02:45:35.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.256+0000 7fef51200640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fef2c065150 0x7fef2c067600 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.256+0000 7fef51200640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fef4c140580 0x7fef4c069860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.256+0000 7fef51200640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef4c1401b0 0x7fef4c0abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.256+0000 7fef51200640 1 --2- 172.21.15.16:0/3215933335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef4c0b1440 0x7fef4c13d180 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:35.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.256+0000 7fef51200640 1 -- 172.21.15.16:0/3215933335 >> 172.21.15.16:0/3215933335 conn(0x7fef4c0cb9b0 msgr2=0x7fef4c0b09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:35.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.256+0000 7fef51200640 1 -- 172.21.15.16:0/3215933335 shutdown_connections 2024-08-17T02:45:35.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:35.256+0000 7fef51200640 1 -- 172.21.15.16:0/3215933335 wait complete. 2024-08-17T02:45:35.272 INFO:teuthology.run_tasks:Running task kclient... 2024-08-17T02:45:35.302 INFO:tasks.kclient:Mounting kernel clients... 2024-08-17T02:45:35.303 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-08-17T02:45:35.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:35.303 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-08-17T02:45:35.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:35.340 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-08-17T02:45:35.438 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-08-17T02:45:35.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:45:35.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:35.441 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-08-17T02:45:35.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:35.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-08-17T02:45:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-08-17T02:45:35.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:45:35.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:35.578 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-08-17T02:45:35.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:35.617 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2024-08-17T02:45:35.721 INFO:teuthology.orchestra.run.smithi142.stderr:Cannot find device "ceph-brx" 2024-08-17T02:45:35.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:45:35.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:35.725 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-08-17T02:45:35.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:35.763 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2024-08-17T02:45:35.861 INFO:teuthology.orchestra.run.smithi142.stderr:Cannot find device "ceph-brx" 2024-08-17T02:45:35.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:45:35.864 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-08-17T02:45:35.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-17T02:45:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.015+0000 7f5d2a600640 1 Processor -- start 2024-08-17T02:45:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.015+0000 7f5d2a600640 1 -- start start 2024-08-17T02:45:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.016+0000 7f5d2a600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5d24155a20 0x7f5d240adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.016+0000 7f5d2a600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d24156670 con 0x7f5d2406df00 2024-08-17T02:45:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.016+0000 7f5d2a600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2406e5e0 con 0x7f5d24155df0 2024-08-17T02:45:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.016+0000 7f5d2a600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d240b0350 con 0x7f5d24155a20 2024-08-17T02:45:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.016+0000 7f5d23e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5d2406df00 0x7f5d24156b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47414/0 (socket says 172.21.15.16:47414) 2024-08-17T02:45:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.016+0000 7f5d23e00640 1 -- 172.21.15.16:0/1015954894 learned_addr learned my addr 172.21.15.16:0/1015954894 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d23400640 1 --2- 172.21.15.16:0/1015954894 >> v2:172.21.15.142:3300/0 conn(0x7f5d24155a20 0x7f5d240adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255210743 0 0) 0x7f5d24156670 con 0x7f5d2406df00 2024-08-17T02:45:36.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d080034e0 con 0x7f5d2406df00 2024-08-17T02:45:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3343113228 0 0) 0x7f5d080034e0 con 0x7f5d2406df00 2024-08-17T02:45:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 >> v2:172.21.15.142:3300/0 conn(0x7f5d24155a20 msgr2=0x7f5d240adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 --2- 172.21.15.16:0/1015954894 >> v2:172.21.15.142:3300/0 conn(0x7f5d24155a20 0x7f5d240adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 >> v1:172.21.15.142:6789/0 conn(0x7f5d24155df0 legacy=0x7f5d2406d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2406e8c0 con 0x7f5d2406df00 2024-08-17T02:45:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7f5d1c0017f0 con 0x7f5d2406df00 2024-08-17T02:45:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 >> v1:172.21.15.16:6789/0 conn(0x7f5d2406df00 legacy=0x7f5d24156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 --2- 172.21.15.16:0/1015954894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 0x7f5d08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 --2- 172.21.15.16:0/1015954894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d08004780 0x7f5d08004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 --2- 172.21.15.16:0/1015954894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d08005130 0x7f5d0800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0800c0a0 con 0x7f5d080039d0 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0800c210 con 0x7f5d08005130 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.017+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0800c3d0 con 0x7f5d08004780 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.018+0000 7f5d23400640 1 --2- 172.21.15.16:0/1015954894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 0x7f5d08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.018+0000 7f5d1be00640 1 --2- 172.21.15.16:0/1015954894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d08004780 0x7f5d08004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.018+0000 7f5d23e00640 1 --2- 172.21.15.16:0/1015954894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d08005130 0x7f5d0800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.018+0000 7f5d23400640 1 -- 172.21.15.16:0/1015954894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d08004780 msgr2=0x7f5d08004bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.018+0000 7f5d23400640 1 --2- 172.21.15.16:0/1015954894 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d08004780 0x7f5d08004bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.018+0000 7f5d23400640 1 -- 172.21.15.16:0/1015954894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d08005130 msgr2=0x7f5d0800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.018+0000 7f5d23400640 1 --2- 172.21.15.16:0/1015954894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d08005130 0x7f5d0800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.018+0000 7f5d23400640 1 -- 172.21.15.16:0/1015954894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d080034e0 con 0x7f5d080039d0 2024-08-17T02:45:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.018+0000 7f5d23e00640 1 -- 172.21.15.16:0/1015954894 reap_dead start 2024-08-17T02:45:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.019+0000 7f5d23400640 1 --2- 172.21.15.16:0/1015954894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 0x7f5d08003de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009b70 tx=0x7f5d140090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d75e34d5764bcc6 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.019+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14031400 con 0x7f5d080039d0 2024-08-17T02:45:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.020+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1402d050 con 0x7f5d080039d0 2024-08-17T02:45:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.020+0000 7f5d22a00640 1 -- 172.21.15.16:0/1015954894 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14031790 con 0x7f5d080039d0 2024-08-17T02:45:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.020+0000 7f5d2a600640 1 -- 172.21.15.16:0/1015954894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 msgr2=0x7f5d08003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.020+0000 7f5d2a600640 1 --2- 172.21.15.16:0/1015954894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 0x7f5d08003de0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009b70 tx=0x7f5d140090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.020+0000 7f5d2a600640 1 -- 172.21.15.16:0/1015954894 shutdown_connections 2024-08-17T02:45:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.020+0000 7f5d2a600640 1 --2- 172.21.15.16:0/1015954894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 0x7f5d08003de0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.020+0000 7f5d2a600640 1 -- 172.21.15.16:0/1015954894 >> 172.21.15.16:0/1015954894 conn(0x7f5d240cb9b0 msgr2=0x7f5d240cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.020+0000 7f5d2a600640 1 -- 172.21.15.16:0/1015954894 shutdown_connections 2024-08-17T02:45:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.020+0000 7f5d2a600640 1 -- 172.21.15.16:0/1015954894 wait complete. 2024-08-17T02:45:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.021+0000 7f5d2a600640 1 Processor -- start 2024-08-17T02:45:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.022+0000 7f5d2a600640 1 -- start start 2024-08-17T02:45:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.022+0000 7f5d2a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 0x7f5d24159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.022+0000 7f5d2a600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2415a4b0 0x7f5d24169100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d2a600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d24169640 0x7f5d24169aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d2a600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d24169fe0 con 0x7f5d080039d0 2024-08-17T02:45:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d2a600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2415af80 con 0x7f5d2415a4b0 2024-08-17T02:45:36.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d2a600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d240aadc0 con 0x7f5d24169640 2024-08-17T02:45:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d23400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 0x7f5d24159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d23400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 0x7f5d24159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41650/0 (socket says 172.21.15.16:41650) 2024-08-17T02:45:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d23400640 1 -- 172.21.15.16:0/1513615814 learned_addr learned my addr 172.21.15.16:0/1513615814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d1be00640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2415a4b0 0x7f5d24169100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d23e00640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d24169640 0x7f5d24169aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d1be00640 1 -- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d24169640 msgr2=0x7f5d24169aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d1be00640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d24169640 0x7f5d24169aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d1be00640 1 -- 172.21.15.16:0/1513615814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 msgr2=0x7f5d24159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d1be00640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 0x7f5d24159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d1be00640 1 -- 172.21.15.16:0/1513615814 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2415a9c0 con 0x7f5d2415a4b0 2024-08-17T02:45:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.023+0000 7f5d23400640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 0x7f5d24159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:36.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.024+0000 7f5d1be00640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2415a4b0 0x7f5d24169100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5d2404c0c0 tx=0x7f5d0c007ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.024+0000 7f5d1aa00640 1 -- 172.21.15.16:0/1513615814 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c012560 con 0x7f5d2415a4b0 2024-08-17T02:45:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.024+0000 7f5d1aa00640 1 -- 172.21.15.16:0/1513615814 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0c0126c0 con 0x7f5d2415a4b0 2024-08-17T02:45:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.024+0000 7f5d1aa00640 1 -- 172.21.15.16:0/1513615814 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c012a60 con 0x7f5d2415a4b0 2024-08-17T02:45:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.024+0000 7f5d2a600640 1 -- 172.21.15.16:0/1513615814 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2415aca0 con 0x7f5d2415a4b0 2024-08-17T02:45:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.024+0000 7f5d2a600640 1 -- 172.21.15.16:0/1513615814 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d240ad1e0 con 0x7f5d2415a4b0 2024-08-17T02:45:36.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.025+0000 7f5d2a600640 1 -- 172.21.15.16:0/1513615814 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d240aaf50 con 0x7f5d2415a4b0 2024-08-17T02:45:36.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.026+0000 7f5d1aa00640 1 -- 172.21.15.16:0/1513615814 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f5d0c01d460 con 0x7f5d2415a4b0 2024-08-17T02:45:36.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.027+0000 7f5d1aa00640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5d00065150 0x7f5d00067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:36.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.027+0000 7f5d1aa00640 1 -- 172.21.15.16:0/1513615814 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5d0c0820d0 con 0x7f5d2415a4b0 2024-08-17T02:45:36.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.027+0000 7f5d23400640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5d00065150 0x7f5d00067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:36.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.028+0000 7f5d23400640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5d00065150 0x7f5d00067600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d14000e50 tx=0x7f5d14035000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:36.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.032+0000 7f5d1aa00640 1 -- 172.21.15.16:0/1513615814 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5d0c0501a0 con 0x7f5d2415a4b0 2024-08-17T02:45:36.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.252+0000 7f5d2a600640 1 -- 172.21.15.16:0/1513615814 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5d24064db0 con 0x7f5d2415a4b0 2024-08-17T02:45:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.253+0000 7f5d1aa00640 1 -- 172.21.15.16:0/1513615814 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5d0c0537c0 con 0x7f5d2415a4b0 2024-08-17T02:45:36.254 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-17T02:45:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.255+0000 7f5d2a600640 1 -- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5d00065150 msgr2=0x7f5d00067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.255+0000 7f5d2a600640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5d00065150 0x7f5d00067600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5d14000e50 tx=0x7f5d14035000 comp rx=0 tx=0).stop 2024-08-17T02:45:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.255+0000 7f5d2a600640 1 -- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2415a4b0 msgr2=0x7f5d24169100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.255+0000 7f5d2a600640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2415a4b0 0x7f5d24169100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5d2404c0c0 tx=0x7f5d0c007ca0 comp rx=0 tx=0).stop 2024-08-17T02:45:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.255+0000 7f5d2a600640 1 -- 172.21.15.16:0/1513615814 shutdown_connections 2024-08-17T02:45:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.255+0000 7f5d2a600640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5d00065150 0x7f5d00067600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.255+0000 7f5d2a600640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5d24169640 0x7f5d24169aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.255+0000 7f5d2a600640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2415a4b0 0x7f5d24169100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.255+0000 7f5d2a600640 1 --2- 172.21.15.16:0/1513615814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d080039d0 0x7f5d24159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.255+0000 7f5d2a600640 1 -- 172.21.15.16:0/1513615814 >> 172.21.15.16:0/1513615814 conn(0x7f5d240cb9b0 msgr2=0x7f5d24157030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.255+0000 7f5d2a600640 1 -- 172.21.15.16:0/1513615814 shutdown_connections 2024-08-17T02:45:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:36.255+0000 7f5d2a600640 1 -- 172.21.15.16:0/1513615814 wait complete. 2024-08-17T02:45:36.272 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-17T02:45:36.272 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-17T02:45:36.272 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-08-17T02:45:36.272 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-17T02:45:36.272 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-17T02:45:36.272 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-17T02:45:36.272 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-17T02:45:36.272 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-17T02:45:36.272 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-17T02:45:36.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:36.273 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-08-17T02:45:36.308 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:45:36.308 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:45:36.308 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:45:36.308 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:36.308 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42642sec preferred_lft 42642sec 2024-08-17T02:45:36.309 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-08-17T02:45:36.310 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:36.310 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:45:36.310 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:36.310 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-08-17T02:45:36.310 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-17T02:45:36.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:36.311 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-08-17T02:45:36.311 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-17T02:45:36.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-08-17T02:45:36.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-08-17T02:45:36.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-08-17T02:45:36.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-17T02:45:36.311 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-17T02:45:36.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:36.675 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-17T02:45:36.732 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-08-17T02:45:36.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:36.735 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-08-17T02:45:36.786 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-08-17T02:45:36.786 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-17T02:45:36.786 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-08-17T02:45:36.787 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-08-17T02:45:36.787 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-17T02:45:36.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:36.787 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-08-17T02:45:36.787 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-17T02:45:36.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-08-17T02:45:36.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-08-17T02:45:36.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-08-17T02:45:36.788 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-17T02:45:37.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:37.335 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-08-17T02:45:37.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:37.374 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-08-17T02:45:37.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:37.452 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-08-17T02:45:37.452 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-17T02:45:37.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-17T02:45:37.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-17T02:45:37.452 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-17T02:45:37.581 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-17T02:45:37.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:37.582 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-08-17T02:45:37.582 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-17T02:45:37.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-17T02:45:37.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-17T02:45:37.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-17T02:45:37.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-17T02:45:37.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-17T02:45:37.582 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-17T02:45:37.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:37.779 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-08-17T02:45:37.779 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-17T02:45:37.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-08-17T02:45:37.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-08-17T02:45:37.779 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-17T02:45:37.896 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:45:37.896 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-17T02:45:37.931 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-17T02:45:37.932 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:45:37.932 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-17T02:45:38.007 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-08-17T02:45:38.007 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-17T02:45:38.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:45:38.135 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:45:38.135 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-08-17T02:45:38.763 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:45:38.764 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-08-17T02:45:38.764 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-08-17T02:45:38.764 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778.cephfs=/ 2024-08-17T02:45:38.764 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.142:3300/172.21.15.142:3301" will pass to kernel 2024-08-17T02:45:38.765 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-08-17T02:45:38.765 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-08-17T02:45:38.765 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-17T02:45:38.765 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-08-17T02:45:38.765 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-17T02:45:38.768 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-17T02:45:38.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:38.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-17T02:45:38.826 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-08-17T02:45:38.826 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-17T02:45:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.975+0000 7fd2a0a00640 1 Processor -- start 2024-08-17T02:45:38.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.976+0000 7fd2a0a00640 1 -- start start 2024-08-17T02:45:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.977+0000 7fd2a0a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29c155900 0x7fd29c155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.977+0000 7fd2a0a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd29c0b0270 0x7fd29c156210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.977+0000 7fd2a0a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd29c157b20 0x7fd29c157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.977+0000 7fd2a0a00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29c157690 con 0x7fd29c0b0270 2024-08-17T02:45:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.977+0000 7fd2a0a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29c156a80 con 0x7fd29c157b20 2024-08-17T02:45:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.977+0000 7fd2a0a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c156d60 con 0x7fd29c155900 2024-08-17T02:45:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.977+0000 7fd299e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd29c0b0270 0x7fd29c156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.977+0000 7fd299e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd29c0b0270 0x7fd29c156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41662/0 (socket says 172.21.15.16:41662) 2024-08-17T02:45:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.977+0000 7fd299e00640 1 -- 172.21.15.16:0/2310487880 learned_addr learned my addr 172.21.15.16:0/2310487880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.977+0000 7fd29a800640 1 --2- 172.21.15.16:0/2310487880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29c155900 0x7fd29c155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.977+0000 7fd29b200640 1 --2- 172.21.15.16:0/2310487880 >> v2:172.21.15.142:3300/0 conn(0x7fd29c157b20 0x7fd29c157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.978+0000 7fd299e00640 1 -- 172.21.15.16:0/2310487880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29c155900 msgr2=0x7fd29c155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.978+0000 7fd299e00640 1 --2- 172.21.15.16:0/2310487880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29c155900 0x7fd29c155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.978+0000 7fd299e00640 1 -- 172.21.15.16:0/2310487880 >> v2:172.21.15.142:3300/0 conn(0x7fd29c157b20 msgr2=0x7fd29c157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.978+0000 7fd299e00640 1 --2- 172.21.15.16:0/2310487880 >> v2:172.21.15.142:3300/0 conn(0x7fd29c157b20 0x7fd29c157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.978+0000 7fd299e00640 1 -- 172.21.15.16:0/2310487880 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c157040 con 0x7fd29c0b0270 2024-08-17T02:45:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.979+0000 7fd299e00640 1 --2- 172.21.15.16:0/2310487880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c0b0270 0x7fd29c156210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0cc860 tx=0x7fd2880091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c3625bdbd29b44 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.979+0000 7fd299400640 1 -- 172.21.15.16:0/2310487880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288031510 con 0x7fd29c0b0270 2024-08-17T02:45:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.979+0000 7fd299400640 1 -- 172.21.15.16:0/2310487880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28802f050 con 0x7fd29c0b0270 2024-08-17T02:45:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.979+0000 7fd299400640 1 -- 172.21.15.16:0/2310487880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2880318e0 con 0x7fd29c0b0270 2024-08-17T02:45:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.979+0000 7fd2a0a00640 1 -- 172.21.15.16:0/2310487880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c0b0270 msgr2=0x7fd29c156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.979+0000 7fd2a0a00640 1 --2- 172.21.15.16:0/2310487880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c0b0270 0x7fd29c156210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0cc860 tx=0x7fd2880091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.980+0000 7fd2a0a00640 1 -- 172.21.15.16:0/2310487880 shutdown_connections 2024-08-17T02:45:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.980+0000 7fd2a0a00640 1 --2- 172.21.15.16:0/2310487880 >> v2:172.21.15.142:3300/0 conn(0x7fd29c157b20 0x7fd29c157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.980+0000 7fd2a0a00640 1 --2- 172.21.15.16:0/2310487880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c0b0270 0x7fd29c156210 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.980+0000 7fd2a0a00640 1 --2- 172.21.15.16:0/2310487880 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29c155900 0x7fd29c155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.980+0000 7fd2a0a00640 1 -- 172.21.15.16:0/2310487880 >> 172.21.15.16:0/2310487880 conn(0x7fd29c067750 msgr2=0x7fd29c0cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.980+0000 7fd2a0a00640 1 -- 172.21.15.16:0/2310487880 shutdown_connections 2024-08-17T02:45:38.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.980+0000 7fd2a0a00640 1 -- 172.21.15.16:0/2310487880 wait complete. 2024-08-17T02:45:38.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.981+0000 7fd2a0a00640 1 Processor -- start 2024-08-17T02:45:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.981+0000 7fd2a0a00640 1 -- start start 2024-08-17T02:45:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.982+0000 7fd2a0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29c0b0270 0x7fd29c0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.982+0000 7fd2a0a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c155900 0x7fd29c1647d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.982+0000 7fd2a0a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29c157b20 0x7fd29c153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.982+0000 7fd2a0a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c159be0 con 0x7fd29c155900 2024-08-17T02:45:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.982+0000 7fd2a0a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c165250 con 0x7fd29c0b0270 2024-08-17T02:45:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.982+0000 7fd2a0a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c165530 con 0x7fd29c157b20 2024-08-17T02:45:38.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.982+0000 7fd299e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c155900 0x7fd29c1647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.982+0000 7fd299e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c155900 0x7fd29c1647d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41670/0 (socket says 172.21.15.16:41670) 2024-08-17T02:45:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.982+0000 7fd299e00640 1 -- 172.21.15.16:0/1419571391 learned_addr learned my addr 172.21.15.16:0/1419571391 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.982+0000 7fd29a800640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29c0b0270 0x7fd29c0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.982+0000 7fd29b200640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29c157b20 0x7fd29c153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.983+0000 7fd299e00640 1 -- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29c157b20 msgr2=0x7fd29c153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.983+0000 7fd299e00640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29c157b20 0x7fd29c153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.983+0000 7fd299e00640 1 -- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29c0b0270 msgr2=0x7fd29c0c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:38.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.983+0000 7fd299e00640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29c0b0270 0x7fd29c0c8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.983+0000 7fd299e00640 1 -- 172.21.15.16:0/1419571391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd288009450 con 0x7fd29c155900 2024-08-17T02:45:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.983+0000 7fd299e00640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c155900 0x7fd29c1647d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd288009940 tx=0x7fd288037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.983+0000 7fd292000640 1 -- 172.21.15.16:0/1419571391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288033860 con 0x7fd29c155900 2024-08-17T02:45:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.983+0000 7fd292000640 1 -- 172.21.15.16:0/1419571391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd288041070 con 0x7fd29c155900 2024-08-17T02:45:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.983+0000 7fd2a0a00640 1 -- 172.21.15.16:0/1419571391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29c159870 con 0x7fd29c155900 2024-08-17T02:45:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.983+0000 7fd292000640 1 -- 172.21.15.16:0/1419571391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd288033c30 con 0x7fd29c155900 2024-08-17T02:45:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.984+0000 7fd2a0a00640 1 -- 172.21.15.16:0/1419571391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29c0c88e0 con 0x7fd29c155900 2024-08-17T02:45:38.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.985+0000 7fd2a0a00640 1 -- 172.21.15.16:0/1419571391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29c159d90 con 0x7fd29c155900 2024-08-17T02:45:38.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.985+0000 7fd292000640 1 -- 172.21.15.16:0/1419571391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fd28803d410 con 0x7fd29c155900 2024-08-17T02:45:38.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.987+0000 7fd292000640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd27c065150 0x7fd27c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:38.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.987+0000 7fd292000640 1 -- 172.21.15.16:0/1419571391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd288043070 con 0x7fd29c155900 2024-08-17T02:45:38.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.988+0000 7fd29a800640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd27c065150 0x7fd27c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:38.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.988+0000 7fd29a800640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd27c065150 0x7fd27c067600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009d20 tx=0x7fd28c0090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:38.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:38.992+0000 7fd292000640 1 -- 172.21.15.16:0/1419571391 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd29c159d90 con 0x7fd29c155900 2024-08-17T02:45:39.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.216+0000 7fd2a0a00640 1 -- 172.21.15.16:0/1419571391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd29c071050 con 0x7fd29c155900 2024-08-17T02:45:39.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.216+0000 7fd292000640 1 -- 172.21.15.16:0/1419571391 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd288073a70 con 0x7fd29c155900 2024-08-17T02:45:39.218 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-17T02:45:39.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.218+0000 7fd2a0a00640 1 -- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd27c065150 msgr2=0x7fd27c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:39.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.218+0000 7fd2a0a00640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd27c065150 0x7fd27c067600 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009d20 tx=0x7fd28c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:39.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.218+0000 7fd2a0a00640 1 -- 172.21.15.16:0/1419571391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c155900 msgr2=0x7fd29c1647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:39.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.218+0000 7fd2a0a00640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c155900 0x7fd29c1647d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd288009940 tx=0x7fd288037000 comp rx=0 tx=0).stop 2024-08-17T02:45:39.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.219+0000 7fd2a0a00640 1 -- 172.21.15.16:0/1419571391 shutdown_connections 2024-08-17T02:45:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.219+0000 7fd2a0a00640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd27c065150 0x7fd27c067600 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.219+0000 7fd2a0a00640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd29c157b20 0x7fd29c153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.219+0000 7fd2a0a00640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c155900 0x7fd29c1647d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.219+0000 7fd2a0a00640 1 --2- 172.21.15.16:0/1419571391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd29c0b0270 0x7fd29c0c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.219+0000 7fd2a0a00640 1 -- 172.21.15.16:0/1419571391 >> 172.21.15.16:0/1419571391 conn(0x7fd29c067750 msgr2=0x7fd29c151820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.219+0000 7fd2a0a00640 1 -- 172.21.15.16:0/1419571391 shutdown_connections 2024-08-17T02:45:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:39.219+0000 7fd2a0a00640 1 -- 172.21.15.16:0/1419571391 wait complete. 2024-08-17T02:45:39.235 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-17T02:45:39.235 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-17T02:45:39.235 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-08-17T02:45:39.235 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-08-17T02:45:39.235 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-08-17T02:45:39.236 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-17T02:45:39.236 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-17T02:45:39.236 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-17T02:45:39.236 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-08-17T02:45:39.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:39.236 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-08-17T02:45:39.273 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:45:39.273 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:45:39.274 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:45:39.274 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.274 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-08-17T02:45:39.274 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:45:39.274 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:39.274 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-08-17T02:45:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:45:39.274 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:39.274 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-08-17T02:45:39.274 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-17T02:45:39.274 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42639sec preferred_lft 42639sec 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether fa:8a:ae:13:08:49 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.275 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::a4bb:7ff:fefc:5693/64 scope link 2024-08-17T02:45:39.276 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.276 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-17T02:45:39.276 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether fa:8a:ae:13:08:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-17T02:45:39.276 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f88a:aeff:fe13:849/64 scope link 2024-08-17T02:45:39.276 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:39.276 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-17T02:45:39.369 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-08-17T02:45:39.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:39.372 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-08-17T02:45:39.410 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-08-17T02:45:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-17T02:45:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-08-17T02:45:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-08-17T02:45:39.411 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-17T02:45:39.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:39.411 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-08-17T02:45:39.411 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-17T02:45:39.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-08-17T02:45:39.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-08-17T02:45:39.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-08-17T02:45:39.412 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-17T02:45:39.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:39.558 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-08-17T02:45:39.593 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-17T02:45:39.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:39.593 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-08-17T02:45:39.667 INFO:teuthology.orchestra.run.smithi016.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-08-17T02:45:39.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:39.669 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-08-17T02:45:39.669 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-17T02:45:39.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-17T02:45:39.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-08-17T02:45:39.669 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-17T02:45:39.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:39.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-08-17T02:45:39.851 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:45:39.852 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:45:39.852 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:45:39.852 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.852 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-08-17T02:45:39.852 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.852 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:45:39.852 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether aa:f3:91:bd:49:a8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-17T02:45:39.852 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-17T02:45:39.852 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.852 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::a8f3:91ff:febd:49a8/64 scope link 2024-08-17T02:45:39.852 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:39.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-08-17T02:45:39.912 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:45:39.912 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:45:39.912 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:45:39.912 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.912 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-08-17T02:45:39.912 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.912 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:45:39.912 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether aa:f3:91:bd:49:a8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-17T02:45:39.912 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-17T02:45:39.912 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.913 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::a8f3:91ff:febd:49a8/64 scope link 2024-08-17T02:45:39.913 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:39.915 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-08-17T02:45:39.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:39.915 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-08-17T02:45:39.915 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-17T02:45:39.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-08-17T02:45:39.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-08-17T02:45:39.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-08-17T02:45:39.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-08-17T02:45:39.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-08-17T02:45:39.916 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-17T02:45:40.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:40.082 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-08-17T02:45:40.082 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-17T02:45:40.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.1 up 2024-08-17T02:45:40.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.1 master ceph-brx 2024-08-17T02:45:40.083 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-17T02:45:40.198 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:45:40.198 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-08-17T02:45:40.234 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-08-17T02:45:40.234 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:45:40.234 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-08-17T02:45:40.309 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-08-17T02:45:40.309 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-17T02:45:40.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:45:40.432 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:45:40.432 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-08-17T02:45:40.459 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:45:40.460 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-08-17T02:45:40.460 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-08-17T02:45:40.460 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 2@4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778.cephfs=/ 2024-08-17T02:45:40.460 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.16:3300/172.21.15.142:3300/172.21.15.142:3301" will pass to kernel 2024-08-17T02:45:40.460 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-08-17T02:45:40.460 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-08-17T02:45:40.460 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-17T02:45:40.460 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-08-17T02:45:40.460 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-17T02:45:40.462 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-17T02:45:40.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:40.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-08-17T02:45:40.518 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-08-17T02:45:40.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-17T02:45:40.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.668+0000 7f5b1b600640 1 Processor -- start 2024-08-17T02:45:40.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.668+0000 7f5b1b600640 1 -- start start 2024-08-17T02:45:40.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.669+0000 7f5b1b600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b14158f80 0x7f5b140ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:40.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.669+0000 7f5b1b600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b140cb560 con 0x7f5b14159350 2024-08-17T02:45:40.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.669+0000 7f5b1b600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b140ce6e0 con 0x7f5b140cf280 2024-08-17T02:45:40.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.669+0000 7f5b1b600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140cf7e0 con 0x7f5b14158f80 2024-08-17T02:45:40.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.670+0000 7f5b13e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5b14159350 0x7f5b140cd8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47418/0 (socket says 172.21.15.16:47418) 2024-08-17T02:45:40.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.670+0000 7f5b13e00640 1 -- 172.21.15.16:0/2478848469 learned_addr learned my addr 172.21.15.16:0/2478848469 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:40.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.670+0000 7f5b18c00640 1 --2- 172.21.15.16:0/2478848469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b14158f80 0x7f5b140ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:40.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.670+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796650362 0 0) 0x7f5b140cb560 con 0x7f5b14159350 2024-08-17T02:45:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.670+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5afc0034e0 con 0x7f5b14159350 2024-08-17T02:45:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.670+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4267216821 0 0) 0x7f5afc0034e0 con 0x7f5b14159350 2024-08-17T02:45:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.670+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b14158f80 msgr2=0x7f5b140ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.670+0000 7f5b13400640 1 --2- 172.21.15.16:0/2478848469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b14158f80 0x7f5b140ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.670+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 >> v1:172.21.15.142:6789/0 conn(0x7f5b140cf280 legacy=0x7f5b140cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.670+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140cfac0 con 0x7f5b14159350 2024-08-17T02:45:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.670+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7f5b00001470 con 0x7f5b14159350 2024-08-17T02:45:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.670+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 >> v1:172.21.15.16:6789/0 conn(0x7f5b14159350 legacy=0x7f5b140cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b13400640 1 --2- 172.21.15.16:0/2478848469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b14158f80 0x7f5afc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b13400640 1 --2- 172.21.15.16:0/2478848469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc003f80 0x7f5afc004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b13400640 1 --2- 172.21.15.16:0/2478848469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5afc004cc0 0x7f5afc005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140cf7e0 con 0x7f5afc003f80 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc00bb60 con 0x7f5afc004cc0 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc00bd20 con 0x7f5b14158f80 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b13e00640 1 --2- 172.21.15.16:0/2478848469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc003f80 0x7f5afc004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b19600640 1 --2- 172.21.15.16:0/2478848469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5afc004cc0 0x7f5afc005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b18c00640 1 --2- 172.21.15.16:0/2478848469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b14158f80 0x7f5afc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b13e00640 1 -- 172.21.15.16:0/2478848469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b14158f80 msgr2=0x7f5afc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b13e00640 1 --2- 172.21.15.16:0/2478848469 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5b14158f80 0x7f5afc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b13e00640 1 -- 172.21.15.16:0/2478848469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5afc004cc0 msgr2=0x7f5afc005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b13e00640 1 --2- 172.21.15.16:0/2478848469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5afc004cc0 0x7f5afc005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.671+0000 7f5b13e00640 1 -- 172.21.15.16:0/2478848469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc0034e0 con 0x7f5afc003f80 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.672+0000 7f5b13e00640 1 --2- 172.21.15.16:0/2478848469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc003f80 0x7f5afc004780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5b14159760 tx=0x7f5b00001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b02e59c4a57df49 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.672+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00005890 con 0x7f5afc003f80 2024-08-17T02:45:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.673+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0002d070 con 0x7f5afc003f80 2024-08-17T02:45:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.673+0000 7f5b13400640 1 -- 172.21.15.16:0/2478848469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00005a50 con 0x7f5afc003f80 2024-08-17T02:45:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.673+0000 7f5b1b600640 1 -- 172.21.15.16:0/2478848469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc003f80 msgr2=0x7f5afc004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.673+0000 7f5b1b600640 1 --2- 172.21.15.16:0/2478848469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc003f80 0x7f5afc004780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5b14159760 tx=0x7f5b00001e50 comp rx=0 tx=0).stop 2024-08-17T02:45:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.673+0000 7f5b19600640 1 -- 172.21.15.16:0/2478848469 reap_dead start 2024-08-17T02:45:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.673+0000 7f5b1b600640 1 -- 172.21.15.16:0/2478848469 shutdown_connections 2024-08-17T02:45:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.673+0000 7f5b1b600640 1 -- 172.21.15.16:0/2478848469 >> 172.21.15.16:0/2478848469 conn(0x7f5b140cfd90 msgr2=0x7f5b140d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.673+0000 7f5b1b600640 1 -- 172.21.15.16:0/2478848469 shutdown_connections 2024-08-17T02:45:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.673+0000 7f5b1b600640 1 -- 172.21.15.16:0/2478848469 wait complete. 2024-08-17T02:45:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.675+0000 7f5b1b600640 1 Processor -- start 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.675+0000 7f5b1b600640 1 -- start start 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b1b600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5afc003f80 0x7f5b140b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b1b600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b140b4c40 0x7f5b140c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b1b600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b141544f0 0x7f5b140c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b1b600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140cadc0 con 0x7f5b141544f0 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b1b600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140b5820 con 0x7f5b140b4c40 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b1b600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1416d380 con 0x7f5afc003f80 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b19600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b141544f0 0x7f5b140c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b19600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b141544f0 0x7f5b140c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41686/0 (socket says 172.21.15.16:41686) 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b19600640 1 -- 172.21.15.16:0/3870136155 learned_addr learned my addr 172.21.15.16:0/3870136155 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b13e00640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b140b4c40 0x7f5b140c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b18c00640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5afc003f80 0x7f5b140b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b19600640 1 -- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5afc003f80 msgr2=0x7f5b140b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b19600640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5afc003f80 0x7f5b140b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b19600640 1 -- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b140b4c40 msgr2=0x7f5b140c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b19600640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b140b4c40 0x7f5b140c4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b19600640 1 -- 172.21.15.16:0/3870136155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1416c770 con 0x7f5b141544f0 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b18c00640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5afc003f80 0x7f5b140b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.676+0000 7f5b13e00640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b140b4c40 0x7f5b140c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.677+0000 7f5b19600640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b141544f0 0x7f5b140c5350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5b1415cc00 tx=0x7f5b0c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.677+0000 7f5b07e00640 1 -- 172.21.15.16:0/3870136155 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c017070 con 0x7f5b141544f0 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.677+0000 7f5b07e00640 1 -- 172.21.15.16:0/3870136155 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c006b10 con 0x7f5b141544f0 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.677+0000 7f5b07e00640 1 -- 172.21.15.16:0/3870136155 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c007020 con 0x7f5b141544f0 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.677+0000 7f5b1b600640 1 -- 172.21.15.16:0/3870136155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1416ca50 con 0x7f5b141544f0 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.677+0000 7f5b1b600640 1 -- 172.21.15.16:0/3870136155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1416d010 con 0x7f5b141544f0 2024-08-17T02:45:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.678+0000 7f5b1b600640 1 -- 172.21.15.16:0/3870136155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b14064db0 con 0x7f5b141544f0 2024-08-17T02:45:40.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.679+0000 7f5b07e00640 1 -- 172.21.15.16:0/3870136155 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f5b0c007410 con 0x7f5b141544f0 2024-08-17T02:45:40.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.680+0000 7f5b07e00640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5af4065150 0x7f5af4067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:40.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.680+0000 7f5b07e00640 1 -- 172.21.15.16:0/3870136155 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b0c0813e0 con 0x7f5b141544f0 2024-08-17T02:45:40.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.680+0000 7f5b18c00640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5af4065150 0x7f5af4067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:40.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.681+0000 7f5b18c00640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5af4065150 0x7f5af4067600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009360 tx=0x7f5b08008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:40.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.684+0000 7f5b07e00640 1 -- 172.21.15.16:0/3870136155 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5b0c04f400 con 0x7f5b141544f0 2024-08-17T02:45:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.908+0000 7f5b1b600640 1 -- 172.21.15.16:0/3870136155 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5b1416d4c0 con 0x7f5b141544f0 2024-08-17T02:45:40.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.908+0000 7f5b07e00640 1 -- 172.21.15.16:0/3870136155 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5b0c052a20 con 0x7f5b141544f0 2024-08-17T02:45:40.910 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-17T02:45:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.910+0000 7f5b1b600640 1 -- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5af4065150 msgr2=0x7f5af4067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.910+0000 7f5b1b600640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5af4065150 0x7f5af4067600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009360 tx=0x7f5b08008040 comp rx=0 tx=0).stop 2024-08-17T02:45:40.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.910+0000 7f5b1b600640 1 -- 172.21.15.16:0/3870136155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b141544f0 msgr2=0x7f5b140c5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.910+0000 7f5b1b600640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b141544f0 0x7f5b140c5350 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5b1415cc00 tx=0x7f5b0c007f90 comp rx=0 tx=0).stop 2024-08-17T02:45:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.910+0000 7f5b1b600640 1 -- 172.21.15.16:0/3870136155 shutdown_connections 2024-08-17T02:45:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.910+0000 7f5b1b600640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5af4065150 0x7f5af4067600 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.910+0000 7f5b1b600640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b141544f0 0x7f5b140c5350 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.910+0000 7f5b1b600640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b140b4c40 0x7f5b140c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.910+0000 7f5b1b600640 1 --2- 172.21.15.16:0/3870136155 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5afc003f80 0x7f5b140b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.910+0000 7f5b1b600640 1 -- 172.21.15.16:0/3870136155 >> 172.21.15.16:0/3870136155 conn(0x7f5b140cfd90 msgr2=0x7f5b1415cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.911+0000 7f5b1b600640 1 -- 172.21.15.16:0/3870136155 shutdown_connections 2024-08-17T02:45:40.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:40.911+0000 7f5b1b600640 1 -- 172.21.15.16:0/3870136155 wait complete. 2024-08-17T02:45:40.926 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-17T02:45:40.926 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-17T02:45:40.926 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-08-17T02:45:40.926 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-08-17T02:45:40.926 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-08-17T02:45:40.926 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-17T02:45:40.926 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-17T02:45:40.926 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-17T02:45:40.927 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-17T02:45:40.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:40.927 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-08-17T02:45:40.967 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:45:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:45:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:45:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-08-17T02:45:40.968 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:40.969 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:45:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-08-17T02:45:40.969 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:45:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:40.969 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-08-17T02:45:40.969 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:45:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:40.970 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-17T02:45:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-17T02:45:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42637sec preferred_lft 42637sec 2024-08-17T02:45:40.970 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-08-17T02:45:40.971 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:40.971 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-17T02:45:40.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:40.971 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-08-17T02:45:40.971 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-08-17T02:45:40.972 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add name ceph-brx type bridge 2024-08-17T02:45:40.972 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip addr flush dev ceph-brx 2024-08-17T02:45:40.972 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set ceph-brx up 2024-08-17T02:45:40.972 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-17T02:45:40.972 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-08-17T02:45:41.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:41.329 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-17T02:45:41.385 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-08-17T02:45:41.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:41.388 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-08-17T02:45:41.449 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-08-17T02:45:41.450 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-17T02:45:41.450 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-17T02:45:41.450 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-17T02:45:41.450 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-17T02:45:41.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:41.450 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-08-17T02:45:41.450 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-08-17T02:45:41.450 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-17T02:45:41.451 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-17T02:45:41.451 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-17T02:45:41.451 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-08-17T02:45:41.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:41.994 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-08-17T02:45:42.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:42.029 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list-id 2024-08-17T02:45:42.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:42.105 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-08-17T02:45:42.105 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-08-17T02:45:42.105 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-17T02:45:42.105 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-08-17T02:45:42.105 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-08-17T02:45:42.230 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-08-17T02:45:42.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:42.230 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-08-17T02:45:42.231 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-08-17T02:45:42.231 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-08-17T02:45:42.231 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-17T02:45:42.231 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-08-17T02:45:42.231 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-08-17T02:45:42.231 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-08-17T02:45:42.231 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-08-17T02:45:42.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:42.415 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-08-17T02:45:42.415 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-08-17T02:45:42.415 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-08-17T02:45:42.415 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set dev brx.0 master ceph-brx 2024-08-17T02:45:42.415 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-08-17T02:45:42.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:45:42.530 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-08-17T02:45:42.564 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-08-17T02:45:42.565 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:45:42.565 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-08-17T02:45:42.639 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-08-17T02:45:42.639 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-17T02:45:42.639 DEBUG:teuthology.orchestra.run.smithi142:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:45:42.777 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:45:42.777 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-08-17T02:45:43.403 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:45:43.403 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-08-17T02:45:43.403 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-08-17T02:45:43.403 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: trying mount with new device syntax: 1@4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778.cephfs=/ 2024-08-17T02:45:43.403 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.142:3300/172.21.15.142:3301" will pass to kernel 2024-08-17T02:45:43.405 INFO:teuthology.orchestra.run.smithi142.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-08-17T02:45:43.406 INFO:teuthology.orchestra.run.smithi142.stdout: You just mounted a file system that supports labels which does not 2024-08-17T02:45:43.406 INFO:teuthology.orchestra.run.smithi142.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-17T02:45:43.406 INFO:teuthology.orchestra.run.smithi142.stdout: applications will generate AVC messages and not be allowed access to 2024-08-17T02:45:43.406 INFO:teuthology.orchestra.run.smithi142.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-17T02:45:43.408 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-17T02:45:43.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:43.408 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-08-17T02:45:43.464 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-08-17T02:45:43.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-17T02:45:43.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.629+0000 7f9a7ec00640 1 Processor -- start 2024-08-17T02:45:43.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.629+0000 7f9a7ec00640 1 -- start start 2024-08-17T02:45:43.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.630+0000 7f9a7ec00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9a78155900 0x7f9a78155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:43.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.630+0000 7f9a7ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a780b0270 0x7f9a78156210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:43.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.630+0000 7f9a7ec00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a78157690 con 0x7f9a78157b20 2024-08-17T02:45:43.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.630+0000 7f9a7ec00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a78156a80 con 0x7f9a78155900 2024-08-17T02:45:43.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.630+0000 7f9a7ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78156d60 con 0x7f9a780b0270 2024-08-17T02:45:43.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.630+0000 7f9a7e200640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9a78157b20 0x7f9a78157fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47422/0 (socket says 172.21.15.16:47422) 2024-08-17T02:45:43.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.630+0000 7f9a7e200640 1 -- 172.21.15.16:0/3825255101 learned_addr learned my addr 172.21.15.16:0/3825255101 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:43.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.630+0000 7f9a7ce00640 1 --2- 172.21.15.16:0/3825255101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a780b0270 0x7f9a78156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:43.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.631+0000 7f9a7d800640 1 --2- 172.21.15.16:0/3825255101 >> v2:172.21.15.142:3300/0 conn(0x7f9a78155900 0x7f9a78155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:43.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.631+0000 7f9a77e00640 1 -- 172.21.15.16:0/3825255101 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246465991 0 0) 0x7f9a78157690 con 0x7f9a78157b20 2024-08-17T02:45:43.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.631+0000 7f9a77e00640 1 -- 172.21.15.16:0/3825255101 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a600034e0 con 0x7f9a78157b20 2024-08-17T02:45:43.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.631+0000 7f9a7ce00640 1 -- 172.21.15.16:0/3825255101 >> v2:172.21.15.142:3300/0 conn(0x7f9a78155900 msgr2=0x7f9a78155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:43.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.631+0000 7f9a7ce00640 1 --2- 172.21.15.16:0/3825255101 >> v2:172.21.15.142:3300/0 conn(0x7f9a78155900 0x7f9a78155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:43.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.631+0000 7f9a7ce00640 1 -- 172.21.15.16:0/3825255101 >> v1:172.21.15.16:6789/0 conn(0x7f9a78157b20 legacy=0x7f9a78157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:43.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.631+0000 7f9a7ce00640 1 -- 172.21.15.16:0/3825255101 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a78157040 con 0x7f9a780b0270 2024-08-17T02:45:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.632+0000 7f9a7ce00640 1 --2- 172.21.15.16:0/3825255101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a780b0270 0x7f9a78156210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009850 tx=0x7f9a640090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed06280d2a4a0005 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.632+0000 7f9a77e00640 1 -- 172.21.15.16:0/3825255101 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6402ad00 con 0x7f9a780b0270 2024-08-17T02:45:43.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.632+0000 7f9a77e00640 1 -- 172.21.15.16:0/3825255101 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6402ae60 con 0x7f9a780b0270 2024-08-17T02:45:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.633+0000 7f9a77e00640 1 -- 172.21.15.16:0/3825255101 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64032670 con 0x7f9a780b0270 2024-08-17T02:45:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.633+0000 7f9a7ec00640 1 -- 172.21.15.16:0/3825255101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a780b0270 msgr2=0x7f9a78156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.633+0000 7f9a7ec00640 1 --2- 172.21.15.16:0/3825255101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a780b0270 0x7f9a78156210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009850 tx=0x7f9a640090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.633+0000 7f9a7ec00640 1 -- 172.21.15.16:0/3825255101 shutdown_connections 2024-08-17T02:45:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.633+0000 7f9a7ec00640 1 --2- 172.21.15.16:0/3825255101 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a780b0270 0x7f9a78156210 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.633+0000 7f9a7ec00640 1 --2- 172.21.15.16:0/3825255101 >> v2:172.21.15.142:3300/0 conn(0x7f9a78155900 0x7f9a78155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.633+0000 7f9a7ec00640 1 -- 172.21.15.16:0/3825255101 >> 172.21.15.16:0/3825255101 conn(0x7f9a78067750 msgr2=0x7f9a780cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.633+0000 7f9a7ec00640 1 -- 172.21.15.16:0/3825255101 shutdown_connections 2024-08-17T02:45:43.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.633+0000 7f9a7ec00640 1 -- 172.21.15.16:0/3825255101 wait complete. 2024-08-17T02:45:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.635+0000 7f9a7ec00640 1 Processor -- start 2024-08-17T02:45:43.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.635+0000 7f9a7ec00640 1 -- start start 2024-08-17T02:45:43.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ec00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a780b0270 0x7f9a7813c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:43.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ec00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a78155900 0x7f9a7814b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a78157b20 0x7f9a78143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ec00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78143ae0 con 0x7f9a78155900 2024-08-17T02:45:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7813c880 con 0x7f9a78157b20 2024-08-17T02:45:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ec00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78142bf0 con 0x7f9a780b0270 2024-08-17T02:45:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a78155900 0x7f9a7814b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a78155900 0x7f9a7814b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41692/0 (socket says 172.21.15.16:41692) 2024-08-17T02:45:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ce00640 1 -- 172.21.15.16:0/2103705511 learned_addr learned my addr 172.21.15.16:0/2103705511 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7d800640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a780b0270 0x7f9a7813c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7e200640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a78157b20 0x7f9a78143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ce00640 1 -- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a780b0270 msgr2=0x7f9a7813c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ce00640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a780b0270 0x7f9a7813c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:43.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ce00640 1 -- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a78157b20 msgr2=0x7f9a78143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:43.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ce00640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a78157b20 0x7f9a78143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:43.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.636+0000 7f9a7ce00640 1 -- 172.21.15.16:0/2103705511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a64009390 con 0x7f9a78155900 2024-08-17T02:45:43.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.637+0000 7f9a7ce00640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a78155900 0x7f9a7814b470 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009360 tx=0x7f9a64032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:43.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.637+0000 7f9a74c00640 1 -- 172.21.15.16:0/2103705511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6403b6e0 con 0x7f9a78155900 2024-08-17T02:45:43.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.637+0000 7f9a74c00640 1 -- 172.21.15.16:0/2103705511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6403b840 con 0x7f9a78155900 2024-08-17T02:45:43.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.637+0000 7f9a74c00640 1 -- 172.21.15.16:0/2103705511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6403bbe0 con 0x7f9a78155900 2024-08-17T02:45:43.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.637+0000 7f9a7ec00640 1 -- 172.21.15.16:0/2103705511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a78142ed0 con 0x7f9a78155900 2024-08-17T02:45:43.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.637+0000 7f9a7ec00640 1 -- 172.21.15.16:0/2103705511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a78143490 con 0x7f9a78155900 2024-08-17T02:45:43.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.638+0000 7f9a7ec00640 1 -- 172.21.15.16:0/2103705511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a78143c90 con 0x7f9a78155900 2024-08-17T02:45:43.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.639+0000 7f9a74c00640 1 -- 172.21.15.16:0/2103705511 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f9a64039040 con 0x7f9a78155900 2024-08-17T02:45:43.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.640+0000 7f9a74c00640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9a58065150 0x7f9a58067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:43.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.640+0000 7f9a74c00640 1 -- 172.21.15.16:0/2103705511 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9a64041030 con 0x7f9a78155900 2024-08-17T02:45:43.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.641+0000 7f9a7d800640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9a58065150 0x7f9a58067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:43.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.641+0000 7f9a7d800640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9a58065150 0x7f9a58067600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009d20 tx=0x7f9a6c0090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:43.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.645+0000 7f9a74c00640 1 -- 172.21.15.16:0/2103705511 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9a78143c90 con 0x7f9a78155900 2024-08-17T02:45:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.865+0000 7f9a7ec00640 1 -- 172.21.15.16:0/2103705511 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9a78071050 con 0x7f9a78155900 2024-08-17T02:45:43.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.865+0000 7f9a74c00640 1 -- 172.21.15.16:0/2103705511 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9a64037070 con 0x7f9a78155900 2024-08-17T02:45:43.867 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-17T02:45:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.867+0000 7f9a7ec00640 1 -- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9a58065150 msgr2=0x7f9a58067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.867+0000 7f9a7ec00640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9a58065150 0x7f9a58067600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009d20 tx=0x7f9a6c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.867+0000 7f9a7ec00640 1 -- 172.21.15.16:0/2103705511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a78155900 msgr2=0x7f9a7814b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.867+0000 7f9a7ec00640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a78155900 0x7f9a7814b470 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009360 tx=0x7f9a64032000 comp rx=0 tx=0).stop 2024-08-17T02:45:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.868+0000 7f9a7ec00640 1 -- 172.21.15.16:0/2103705511 shutdown_connections 2024-08-17T02:45:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.868+0000 7f9a7ec00640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9a58065150 0x7f9a58067600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.868+0000 7f9a7ec00640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a78157b20 0x7f9a78143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.868+0000 7f9a7ec00640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a78155900 0x7f9a7814b470 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.868+0000 7f9a7ec00640 1 --2- 172.21.15.16:0/2103705511 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9a780b0270 0x7f9a7813c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:43.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.868+0000 7f9a7ec00640 1 -- 172.21.15.16:0/2103705511 >> 172.21.15.16:0/2103705511 conn(0x7f9a78067750 msgr2=0x7f9a781530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.868+0000 7f9a7ec00640 1 -- 172.21.15.16:0/2103705511 shutdown_connections 2024-08-17T02:45:43.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:43.868+0000 7f9a7ec00640 1 -- 172.21.15.16:0/2103705511 wait complete. 2024-08-17T02:45:43.885 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-17T02:45:43.885 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-17T02:45:43.885 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-08-17T02:45:43.885 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-08-17T02:45:43.885 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-08-17T02:45:43.885 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-17T02:45:43.885 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-17T02:45:43.885 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-17T02:45:43.885 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-08-17T02:45:43.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:43.886 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-08-17T02:45:43.923 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:45:43.923 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:45:43.923 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:45:43.923 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:43.923 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-08-17T02:45:43.923 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:43.923 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:45:43.923 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:43.923 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42634sec preferred_lft 42634sec 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:45:43.924 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 42:93:50:74:3a:42 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:45:43.925 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-17T02:45:43.925 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:43.925 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::7054:9fff:feff:9dd/64 scope link 2024-08-17T02:45:43.925 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:43.925 INFO:teuthology.orchestra.run.smithi142.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-17T02:45:43.925 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 42:93:50:74:3a:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-17T02:45:43.925 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::4093:50ff:fe74:3a42/64 scope link tentative 2024-08-17T02:45:43.925 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:43.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:43.925 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-17T02:45:44.017 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-08-17T02:45:44.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:44.020 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-08-17T02:45:44.057 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-08-17T02:45:44.057 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-17T02:45:44.057 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-17T02:45:44.058 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-17T02:45:44.058 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-17T02:45:44.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:44.058 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-08-17T02:45:44.058 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-08-17T02:45:44.058 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-17T02:45:44.058 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-17T02:45:44.058 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-17T02:45:44.058 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-08-17T02:45:44.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:44.202 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-08-17T02:45:44.238 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-17T02:45:44.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:44.238 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list-id 2024-08-17T02:45:44.314 INFO:teuthology.orchestra.run.smithi142.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-08-17T02:45:44.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:44.314 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-08-17T02:45:44.314 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-08-17T02:45:44.314 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-17T02:45:44.314 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-08-17T02:45:44.314 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-08-17T02:45:44.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:44.440 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-08-17T02:45:44.495 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:45:44.495 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:45:44.495 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:45:44.495 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:44.496 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-08-17T02:45:44.496 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:44.496 INFO:teuthology.orchestra.run.smithi142.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:45:44.496 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 12:ec:20:56:9d:d8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-17T02:45:44.496 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-17T02:45:44.496 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:44.496 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::10ec:20ff:fe56:9dd8/64 scope link 2024-08-17T02:45:44.496 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:44.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:44.498 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-08-17T02:45:44.553 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:45:44.554 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:45:44.554 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:45:44.554 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:44.554 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-08-17T02:45:44.554 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:44.554 INFO:teuthology.orchestra.run.smithi142.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:45:44.554 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 12:ec:20:56:9d:d8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-08-17T02:45:44.554 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-08-17T02:45:44.554 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:44.554 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::10ec:20ff:fe56:9dd8/64 scope link 2024-08-17T02:45:44.554 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:45:44.558 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-08-17T02:45:44.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:44.559 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-08-17T02:45:44.559 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-08-17T02:45:44.559 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-08-17T02:45:44.559 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-08-17T02:45:44.559 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-08-17T02:45:44.559 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-08-17T02:45:44.559 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-08-17T02:45:44.559 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-08-17T02:45:44.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:44.723 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-08-17T02:45:44.723 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-08-17T02:45:44.723 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.1 up 2024-08-17T02:45:44.723 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set dev brx.1 master ceph-brx 2024-08-17T02:45:44.724 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-08-17T02:45:44.837 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:45:44.837 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-08-17T02:45:44.874 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-08-17T02:45:44.874 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:45:44.875 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-08-17T02:45:44.949 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-08-17T02:45:44.950 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-17T02:45:44.950 DEBUG:teuthology.orchestra.run.smithi142:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:45:45.071 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:45:45.071 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-08-17T02:45:45.098 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:45:45.098 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-08-17T02:45:45.098 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-08-17T02:45:45.098 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: trying mount with new device syntax: 3@4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778.cephfs=/ 2024-08-17T02:45:45.098 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.16:3300/172.21.15.142:3300/172.21.15.142:3301" will pass to kernel 2024-08-17T02:45:45.099 INFO:teuthology.orchestra.run.smithi142.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-08-17T02:45:45.099 INFO:teuthology.orchestra.run.smithi142.stdout: You just mounted a file system that supports labels which does not 2024-08-17T02:45:45.099 INFO:teuthology.orchestra.run.smithi142.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-17T02:45:45.099 INFO:teuthology.orchestra.run.smithi142.stdout: applications will generate AVC messages and not be allowed access to 2024-08-17T02:45:45.099 INFO:teuthology.orchestra.run.smithi142.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-17T02:45:45.102 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-17T02:45:45.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:45.102 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-08-17T02:45:45.157 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-08-17T02:45:45.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.339+0000 7f2d33e00640 1 Processor -- start 2024-08-17T02:45:45.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.339+0000 7f2d33e00640 1 -- start start 2024-08-17T02:45:45.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.340+0000 7f2d33e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d340cc9d0 0x7f2d340ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.340+0000 7f2d33e00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d340cf9d0 con 0x7f2d340cd700 2024-08-17T02:45:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.340+0000 7f2d33e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d340ceb60 con 0x7f2d34158c80 2024-08-17T02:45:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.340+0000 7f2d33e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340cedc0 con 0x7f2d340cc9d0 2024-08-17T02:45:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.340+0000 7f2d32000640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2d340cd700 0x7f2d340acfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52756/0 (socket says 172.21.15.16:52756) 2024-08-17T02:45:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.340+0000 7f2d32000640 1 -- 172.21.15.16:0/360230105 learned_addr learned my addr 172.21.15.16:0/360230105 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:45.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.340+0000 7f2d33400640 1 --2- 172.21.15.16:0/360230105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d340cc9d0 0x7f2d340ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.341+0000 7f2d31600640 1 -- 172.21.15.16:0/360230105 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289163076 0 0) 0x7f2d340cf9d0 con 0x7f2d340cd700 2024-08-17T02:45:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.341+0000 7f2d31600640 1 -- 172.21.15.16:0/360230105 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d180034e0 con 0x7f2d340cd700 2024-08-17T02:45:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.341+0000 7f2d33400640 1 -- 172.21.15.16:0/360230105 >> v1:172.21.15.142:6789/0 conn(0x7f2d34158c80 legacy=0x7f2d340cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.341+0000 7f2d31600640 1 -- 172.21.15.16:0/360230105 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338482924 0 0) 0x7f2d340ceb60 con 0x7f2d34158c80 2024-08-17T02:45:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.341+0000 7f2d33400640 1 -- 172.21.15.16:0/360230105 >> v1:172.21.15.16:6789/0 conn(0x7f2d340cd700 legacy=0x7f2d340acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.341+0000 7f2d33400640 1 -- 172.21.15.16:0/360230105 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d340cf0a0 con 0x7f2d340cc9d0 2024-08-17T02:45:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.342+0000 7f2d33400640 1 --2- 172.21.15.16:0/360230105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d340cc9d0 0x7f2d340ce620 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009e90 tx=0x7f2d2c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8fca5ec21f37b57 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.342+0000 7f2d31600640 1 -- 172.21.15.16:0/360230105 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c007550 con 0x7f2d340cc9d0 2024-08-17T02:45:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.342+0000 7f2d31600640 1 -- 172.21.15.16:0/360230105 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2c02d050 con 0x7f2d340cc9d0 2024-08-17T02:45:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.342+0000 7f2d31600640 1 -- 172.21.15.16:0/360230105 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c0078e0 con 0x7f2d340cc9d0 2024-08-17T02:45:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.343+0000 7f2d33e00640 1 -- 172.21.15.16:0/360230105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d340cc9d0 msgr2=0x7f2d340ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.343+0000 7f2d33e00640 1 --2- 172.21.15.16:0/360230105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d340cc9d0 0x7f2d340ce620 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009e90 tx=0x7f2d2c009780 comp rx=0 tx=0).stop 2024-08-17T02:45:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.343+0000 7f2d33e00640 1 -- 172.21.15.16:0/360230105 shutdown_connections 2024-08-17T02:45:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.343+0000 7f2d33e00640 1 --2- 172.21.15.16:0/360230105 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d340cc9d0 0x7f2d340ce620 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.343+0000 7f2d33e00640 1 -- 172.21.15.16:0/360230105 >> 172.21.15.16:0/360230105 conn(0x7f2d34098e30 msgr2=0x7f2d3406eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.343+0000 7f2d33e00640 1 -- 172.21.15.16:0/360230105 shutdown_connections 2024-08-17T02:45:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.343+0000 7f2d33e00640 1 -- 172.21.15.16:0/360230105 wait complete. 2024-08-17T02:45:45.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.345+0000 7f2d33e00640 1 Processor -- start 2024-08-17T02:45:45.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.345+0000 7f2d33e00640 1 -- start start 2024-08-17T02:45:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d33e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d340cc9d0 0x7f2d3416b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d33e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d340cd700 0x7f2d340b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:45.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d33e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34158c80 0x7f2d34067b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d33e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3416be40 con 0x7f2d34158c80 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d33e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340af5a0 con 0x7f2d340cc9d0 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d33e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d340af270 con 0x7f2d340cd700 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d32a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d340cc9d0 0x7f2d3416b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d33400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34158c80 0x7f2d34067b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d32a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d340cc9d0 0x7f2d3416b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:42656/0 (socket says 172.21.15.16:42656) 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d32a00640 1 -- 172.21.15.16:0/1453722367 learned_addr learned my addr 172.21.15.16:0/1453722367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d32000640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d340cd700 0x7f2d340b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d33400640 1 -- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d340cd700 msgr2=0x7f2d340b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d33400640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d340cd700 0x7f2d340b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d33400640 1 -- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d340cc9d0 msgr2=0x7f2d3416b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d33400640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d340cc9d0 0x7f2d3416b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d33400640 1 -- 172.21.15.16:0/1453722367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c009a20 con 0x7f2d34158c80 2024-08-17T02:45:45.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.346+0000 7f2d32a00640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d340cc9d0 0x7f2d3416b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:45.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.347+0000 7f2d33400640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34158c80 0x7f2d34067b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c031000 tx=0x7f2d2c037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:45.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.347+0000 7f2d2a000640 1 -- 172.21.15.16:0/1453722367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c0318b0 con 0x7f2d34158c80 2024-08-17T02:45:45.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.347+0000 7f2d2a000640 1 -- 172.21.15.16:0/1453722367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2c040070 con 0x7f2d34158c80 2024-08-17T02:45:45.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.347+0000 7f2d2a000640 1 -- 172.21.15.16:0/1453722367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c031c60 con 0x7f2d34158c80 2024-08-17T02:45:45.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.347+0000 7f2d33e00640 1 -- 172.21.15.16:0/1453722367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d340b1c30 con 0x7f2d34158c80 2024-08-17T02:45:45.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.347+0000 7f2d33e00640 1 -- 172.21.15.16:0/1453722367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d340b21f0 con 0x7f2d34158c80 2024-08-17T02:45:45.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.348+0000 7f2d33e00640 1 -- 172.21.15.16:0/1453722367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d34071050 con 0x7f2d34158c80 2024-08-17T02:45:45.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.349+0000 7f2d2a000640 1 -- 172.21.15.16:0/1453722367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f2d2c042030 con 0x7f2d34158c80 2024-08-17T02:45:45.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.350+0000 7f2d2a000640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2d14065150 0x7f2d14067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:45.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.350+0000 7f2d2a000640 1 -- 172.21.15.16:0/1453722367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2d2c0a3d10 con 0x7f2d34158c80 2024-08-17T02:45:45.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.350+0000 7f2d32a00640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2d14065150 0x7f2d14067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:45.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.351+0000 7f2d32a00640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2d14065150 0x7f2d14067600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2d34156c80 tx=0x7f2d24009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:45.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.354+0000 7f2d2a000640 1 -- 172.21.15.16:0/1453722367 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2d2c0a9020 con 0x7f2d34158c80 2024-08-17T02:45:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.537+0000 7f2d33e00640 1 -- 172.21.15.16:0/1453722367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d340cbaf0 con 0x7f2d34158c80 2024-08-17T02:45:45.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.538+0000 7f2d2a000640 1 -- 172.21.15.16:0/1453722367 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f2d2c071de0 con 0x7f2d34158c80 2024-08-17T02:45:45.540 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:45.540 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:45.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.540+0000 7f2d33e00640 1 -- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2d14065150 msgr2=0x7f2d14067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.540+0000 7f2d33e00640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2d14065150 0x7f2d14067600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2d34156c80 tx=0x7f2d24009040 comp rx=0 tx=0).stop 2024-08-17T02:45:45.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.540+0000 7f2d33e00640 1 -- 172.21.15.16:0/1453722367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34158c80 msgr2=0x7f2d34067b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.540+0000 7f2d33e00640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34158c80 0x7f2d34067b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c031000 tx=0x7f2d2c037000 comp rx=0 tx=0).stop 2024-08-17T02:45:45.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.540+0000 7f2d33e00640 1 -- 172.21.15.16:0/1453722367 shutdown_connections 2024-08-17T02:45:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.540+0000 7f2d33e00640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2d14065150 0x7f2d14067600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.540+0000 7f2d33e00640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d34158c80 0x7f2d34067b50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.540+0000 7f2d33e00640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2d340cd700 0x7f2d340b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.540+0000 7f2d33e00640 1 --2- 172.21.15.16:0/1453722367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d340cc9d0 0x7f2d3416b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.540+0000 7f2d33e00640 1 -- 172.21.15.16:0/1453722367 >> 172.21.15.16:0/1453722367 conn(0x7f2d34098e30 msgr2=0x7f2d34156cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.540+0000 7f2d33e00640 1 -- 172.21.15.16:0/1453722367 shutdown_connections 2024-08-17T02:45:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.540+0000 7f2d33e00640 1 -- 172.21.15.16:0/1453722367 wait complete. 2024-08-17T02:45:45.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:45:14.291903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:45:18.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:45:18.430718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:45:45.557 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:45:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.711+0000 7fb0e6600640 1 Processor -- start 2024-08-17T02:45:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.711+0000 7fb0e6600640 1 -- start start 2024-08-17T02:45:45.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.712+0000 7fb0e6600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0e0157e10 0x7fb0e00b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:45.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.712+0000 7fb0e6600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0e00adec0 0x7fb0e015a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:45.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.712+0000 7fb0e6600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e0157070 con 0x7fb0e00b0a80 2024-08-17T02:45:45.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.712+0000 7fb0e6600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e00b1390 con 0x7fb0e0157e10 2024-08-17T02:45:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.712+0000 7fb0e6600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e015b410 con 0x7fb0e00adec0 2024-08-17T02:45:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.713+0000 7fb0dea00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0e0157e10 0x7fb0e00b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.713+0000 7fb0dea00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0e0157e10 0x7fb0e00b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36354/0 (socket says 172.21.15.16:36354) 2024-08-17T02:45:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.713+0000 7fb0dea00640 1 -- 172.21.15.16:0/2691735435 learned_addr learned my addr 172.21.15.16:0/2691735435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.713+0000 7fb0dfe00640 1 --2- 172.21.15.16:0/2691735435 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0e00adec0 0x7fb0e015a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.713+0000 7fb0dea00640 1 -- 172.21.15.16:0/2691735435 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0e00adec0 msgr2=0x7fb0e015a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.713+0000 7fb0dea00640 1 --2- 172.21.15.16:0/2691735435 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0e00adec0 0x7fb0e015a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.713+0000 7fb0dea00640 1 -- 172.21.15.16:0/2691735435 >> v1:172.21.15.142:6789/0 conn(0x7fb0e00b0a80 legacy=0x7fb0e00ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.713+0000 7fb0dea00640 1 -- 172.21.15.16:0/2691735435 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e0157330 con 0x7fb0e0157e10 2024-08-17T02:45:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.714+0000 7fb0dea00640 1 --2- 172.21.15.16:0/2691735435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e0157e10 0x7fb0e00b0e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009910 tx=0x7fb0cc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3696eb1481ec2cb8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.714+0000 7fb0de000640 1 -- 172.21.15.16:0/2691735435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc02add0 con 0x7fb0e0157e10 2024-08-17T02:45:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.714+0000 7fb0de000640 1 -- 172.21.15.16:0/2691735435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc032440 con 0x7fb0e0157e10 2024-08-17T02:45:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.714+0000 7fb0de000640 1 -- 172.21.15.16:0/2691735435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc032820 con 0x7fb0e0157e10 2024-08-17T02:45:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.715+0000 7fb0e6600640 1 -- 172.21.15.16:0/2691735435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e0157e10 msgr2=0x7fb0e00b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.715+0000 7fb0e6600640 1 --2- 172.21.15.16:0/2691735435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e0157e10 0x7fb0e00b0e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009910 tx=0x7fb0cc0091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.715+0000 7fb0e6600640 1 -- 172.21.15.16:0/2691735435 shutdown_connections 2024-08-17T02:45:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.715+0000 7fb0e6600640 1 --2- 172.21.15.16:0/2691735435 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0e00adec0 0x7fb0e015a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.715+0000 7fb0e6600640 1 --2- 172.21.15.16:0/2691735435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e0157e10 0x7fb0e00b0e50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.715+0000 7fb0e6600640 1 -- 172.21.15.16:0/2691735435 >> 172.21.15.16:0/2691735435 conn(0x7fb0e0067750 msgr2=0x7fb0e00cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.715+0000 7fb0e6600640 1 -- 172.21.15.16:0/2691735435 shutdown_connections 2024-08-17T02:45:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.715+0000 7fb0e6600640 1 -- 172.21.15.16:0/2691735435 wait complete. 2024-08-17T02:45:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.716+0000 7fb0e6600640 1 Processor -- start 2024-08-17T02:45:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.717+0000 7fb0e6600640 1 -- start start 2024-08-17T02:45:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.717+0000 7fb0e6600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0e00adec0 0x7fb0e00c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.717+0000 7fb0e6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e00b0a80 0x7fb0e0150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.717+0000 7fb0e6600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0e0157e10 0x7fb0e0169520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.717+0000 7fb0e6600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e0150ba0 con 0x7fb0e00b0a80 2024-08-17T02:45:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.717+0000 7fb0e6600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00c0f10 con 0x7fb0e00adec0 2024-08-17T02:45:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.717+0000 7fb0e6600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00c11f0 con 0x7fb0e0157e10 2024-08-17T02:45:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.718+0000 7fb0dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e00b0a80 0x7fb0e0150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.718+0000 7fb0df400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0e00adec0 0x7fb0e00c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.718+0000 7fb0dfe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0e0157e10 0x7fb0e0169520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.718+0000 7fb0dea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e00b0a80 0x7fb0e0150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36358/0 (socket says 172.21.15.16:36358) 2024-08-17T02:45:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.718+0000 7fb0dea00640 1 -- 172.21.15.16:0/725260354 learned_addr learned my addr 172.21.15.16:0/725260354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.718+0000 7fb0dea00640 1 -- 172.21.15.16:0/725260354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0e0157e10 msgr2=0x7fb0e0169520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.718+0000 7fb0dea00640 1 --2- 172.21.15.16:0/725260354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0e0157e10 0x7fb0e0169520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.718+0000 7fb0dea00640 1 -- 172.21.15.16:0/725260354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0e00adec0 msgr2=0x7fb0e00c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.718+0000 7fb0dea00640 1 --2- 172.21.15.16:0/725260354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0e00adec0 0x7fb0e00c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.718+0000 7fb0dea00640 1 -- 172.21.15.16:0/725260354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc009450 con 0x7fb0e00b0a80 2024-08-17T02:45:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.719+0000 7fb0dea00640 1 --2- 172.21.15.16:0/725260354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e00b0a80 0x7fb0e0150660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009210 tx=0x7fb0cc032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.719+0000 7fb0d6a00640 1 -- 172.21.15.16:0/725260354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc000d10 con 0x7fb0e00b0a80 2024-08-17T02:45:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.719+0000 7fb0d6a00640 1 -- 172.21.15.16:0/725260354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc031de0 con 0x7fb0e00b0a80 2024-08-17T02:45:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.719+0000 7fb0d6a00640 1 -- 172.21.15.16:0/725260354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc03d670 con 0x7fb0e00b0a80 2024-08-17T02:45:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.719+0000 7fb0e6600640 1 -- 172.21.15.16:0/725260354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e00c14d0 con 0x7fb0e00b0a80 2024-08-17T02:45:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.719+0000 7fb0e6600640 1 -- 172.21.15.16:0/725260354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e00c1a90 con 0x7fb0e00b0a80 2024-08-17T02:45:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.720+0000 7fb0e6600640 1 -- 172.21.15.16:0/725260354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e0064db0 con 0x7fb0e00b0a80 2024-08-17T02:45:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.721+0000 7fb0d6a00640 1 -- 172.21.15.16:0/725260354 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fb0cc039040 con 0x7fb0e00b0a80 2024-08-17T02:45:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.722+0000 7fb0d6a00640 1 --2- 172.21.15.16:0/725260354 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb0c0065150 0x7fb0c0067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.722+0000 7fb0d6a00640 1 -- 172.21.15.16:0/725260354 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb0cc0a2410 con 0x7fb0e00b0a80 2024-08-17T02:45:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.722+0000 7fb0df400640 1 --2- 172.21.15.16:0/725260354 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb0c0065150 0x7fb0c0067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.723+0000 7fb0df400640 1 --2- 172.21.15.16:0/725260354 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb0c0065150 0x7fb0c0067600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0007fd0 tx=0x7fb0d0007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.726+0000 7fb0d6a00640 1 -- 172.21.15.16:0/725260354 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb0cc0704e0 con 0x7fb0e00b0a80 2024-08-17T02:45:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.908+0000 7fb0e6600640 1 -- 172.21.15.16:0/725260354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0e01563d0 con 0x7fb0e00b0a80 2024-08-17T02:45:45.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.909+0000 7fb0d6a00640 1 -- 172.21.15.16:0/725260354 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0cc037020 con 0x7fb0e00b0a80 2024-08-17T02:45:45.910 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-17T02:45:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.911+0000 7fb0e6600640 1 -- 172.21.15.16:0/725260354 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb0c0065150 msgr2=0x7fb0c0067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.911+0000 7fb0e6600640 1 --2- 172.21.15.16:0/725260354 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb0c0065150 0x7fb0c0067600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0007fd0 tx=0x7fb0d0007e30 comp rx=0 tx=0).stop 2024-08-17T02:45:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.911+0000 7fb0e6600640 1 -- 172.21.15.16:0/725260354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e00b0a80 msgr2=0x7fb0e0150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:45.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.911+0000 7fb0e6600640 1 --2- 172.21.15.16:0/725260354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e00b0a80 0x7fb0e0150660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009210 tx=0x7fb0cc032000 comp rx=0 tx=0).stop 2024-08-17T02:45:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.911+0000 7fb0e6600640 1 -- 172.21.15.16:0/725260354 shutdown_connections 2024-08-17T02:45:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.911+0000 7fb0e6600640 1 --2- 172.21.15.16:0/725260354 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb0c0065150 0x7fb0c0067600 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.911+0000 7fb0e6600640 1 --2- 172.21.15.16:0/725260354 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb0e0157e10 0x7fb0e0169520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.911+0000 7fb0e6600640 1 --2- 172.21.15.16:0/725260354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0e00b0a80 0x7fb0e0150660 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.911+0000 7fb0e6600640 1 --2- 172.21.15.16:0/725260354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0e00adec0 0x7fb0e00c08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.911+0000 7fb0e6600640 1 -- 172.21.15.16:0/725260354 >> 172.21.15.16:0/725260354 conn(0x7fb0e0067750 msgr2=0x7fb0e014d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.911+0000 7fb0e6600640 1 -- 172.21.15.16:0/725260354 shutdown_connections 2024-08-17T02:45:45.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:45.911+0000 7fb0e6600640 1 -- 172.21.15.16:0/725260354 wait complete. 2024-08-17T02:45:45.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:45:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.081+0000 7fc539200640 1 Processor -- start 2024-08-17T02:45:46.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.081+0000 7fc539200640 1 -- start start 2024-08-17T02:45:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.082+0000 7fc539200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc5340c92e0 0x7fc534163d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.082+0000 7fc539200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc53406d790 con 0x7fc534158ce0 2024-08-17T02:45:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.082+0000 7fc539200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc53406cb80 con 0x7fc5341590b0 2024-08-17T02:45:46.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.082+0000 7fc539200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53406ce60 con 0x7fc5340c92e0 2024-08-17T02:45:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.082+0000 7fc531e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc534158ce0 0x7fc53406a1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52768/0 (socket says 172.21.15.16:52768) 2024-08-17T02:45:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.082+0000 7fc531e00640 1 -- 172.21.15.16:0/3363711079 learned_addr learned my addr 172.21.15.16:0/3363711079 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.082+0000 7fc532800640 1 --2- 172.21.15.16:0/3363711079 >> v2:172.21.15.142:3300/0 conn(0x7fc5340c92e0 0x7fc534163d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.083+0000 7fc531400640 1 -- 172.21.15.16:0/3363711079 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774690200 0 0) 0x7fc53406d790 con 0x7fc534158ce0 2024-08-17T02:45:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.083+0000 7fc531400640 1 -- 172.21.15.16:0/3363711079 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5180034e0 con 0x7fc534158ce0 2024-08-17T02:45:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.083+0000 7fc532800640 1 -- 172.21.15.16:0/3363711079 >> v1:172.21.15.142:6789/0 conn(0x7fc5341590b0 legacy=0x7fc5340c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.083+0000 7fc532800640 1 -- 172.21.15.16:0/3363711079 >> v1:172.21.15.16:6789/0 conn(0x7fc534158ce0 legacy=0x7fc53406a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.083+0000 7fc532800640 1 -- 172.21.15.16:0/3363711079 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53406d140 con 0x7fc5340c92e0 2024-08-17T02:45:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.084+0000 7fc532800640 1 --2- 172.21.15.16:0/3363711079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5340c92e0 0x7fc534163d60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009e60 tx=0x7fc52c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87d7e273bd4ae6c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.084+0000 7fc531400640 1 -- 172.21.15.16:0/3363711079 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c007550 con 0x7fc5340c92e0 2024-08-17T02:45:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.084+0000 7fc531400640 1 -- 172.21.15.16:0/3363711079 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c02d050 con 0x7fc5340c92e0 2024-08-17T02:45:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.084+0000 7fc531400640 1 -- 172.21.15.16:0/3363711079 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c007900 con 0x7fc5340c92e0 2024-08-17T02:45:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.085+0000 7fc539200640 1 -- 172.21.15.16:0/3363711079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5340c92e0 msgr2=0x7fc534163d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.085+0000 7fc539200640 1 --2- 172.21.15.16:0/3363711079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5340c92e0 0x7fc534163d60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009e60 tx=0x7fc52c009780 comp rx=0 tx=0).stop 2024-08-17T02:45:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.085+0000 7fc539200640 1 -- 172.21.15.16:0/3363711079 shutdown_connections 2024-08-17T02:45:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.085+0000 7fc539200640 1 --2- 172.21.15.16:0/3363711079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5340c92e0 0x7fc534163d60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.085+0000 7fc539200640 1 -- 172.21.15.16:0/3363711079 >> 172.21.15.16:0/3363711079 conn(0x7fc534067750 msgr2=0x7fc5340c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.085+0000 7fc539200640 1 -- 172.21.15.16:0/3363711079 shutdown_connections 2024-08-17T02:45:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.085+0000 7fc539200640 1 -- 172.21.15.16:0/3363711079 wait complete. 2024-08-17T02:45:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.086+0000 7fc539200640 1 Processor -- start 2024-08-17T02:45:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.086+0000 7fc539200640 1 -- start start 2024-08-17T02:45:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.087+0000 7fc539200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5340c92e0 0x7fc5340ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.087+0000 7fc539200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc534158ce0 0x7fc534147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc539200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5341590b0 0x7fc534149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc539200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5340ad7b0 con 0x7fc534158ce0 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc539200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53414c110 con 0x7fc5340c92e0 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc539200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53414b7e0 con 0x7fc5341590b0 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc52be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc534158ce0 0x7fc534147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc531e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5340c92e0 0x7fc5340ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc531e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5340c92e0 0x7fc5340ad270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:42668/0 (socket says 172.21.15.16:42668) 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc531e00640 1 -- 172.21.15.16:0/3283744554 learned_addr learned my addr 172.21.15.16:0/3283744554 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc532800640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5341590b0 0x7fc534149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc531e00640 1 -- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5341590b0 msgr2=0x7fc534149a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc531e00640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5341590b0 0x7fc534149a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc531e00640 1 -- 172.21.15.16:0/3283744554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc534158ce0 msgr2=0x7fc534147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc531e00640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc534158ce0 0x7fc534147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc531e00640 1 -- 172.21.15.16:0/3283744554 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52c009a20 con 0x7fc5340c92e0 2024-08-17T02:45:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.088+0000 7fc52be00640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc534158ce0 0x7fc534147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.089+0000 7fc531e00640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5340c92e0 0x7fc5340ad270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc52400c5b0 tx=0x7fc524007d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.089+0000 7fc529600640 1 -- 172.21.15.16:0/3283744554 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc524005aa0 con 0x7fc5340c92e0 2024-08-17T02:45:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.089+0000 7fc529600640 1 -- 172.21.15.16:0/3283744554 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc524005c00 con 0x7fc5340c92e0 2024-08-17T02:45:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.089+0000 7fc539200640 1 -- 172.21.15.16:0/3283744554 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53414bac0 con 0x7fc5340c92e0 2024-08-17T02:45:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.089+0000 7fc529600640 1 -- 172.21.15.16:0/3283744554 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5240070f0 con 0x7fc5340c92e0 2024-08-17T02:45:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.089+0000 7fc539200640 1 -- 172.21.15.16:0/3283744554 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5341484c0 con 0x7fc5340c92e0 2024-08-17T02:45:46.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.090+0000 7fc539200640 1 -- 172.21.15.16:0/3283744554 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc53414c3f0 con 0x7fc5340c92e0 2024-08-17T02:45:46.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.091+0000 7fc529600640 1 -- 172.21.15.16:0/3283744554 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fc524007520 con 0x7fc5340c92e0 2024-08-17T02:45:46.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.092+0000 7fc529600640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc510065180 0x7fc510067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.092+0000 7fc529600640 1 -- 172.21.15.16:0/3283744554 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc524081de0 con 0x7fc5340c92e0 2024-08-17T02:45:46.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.092+0000 7fc52be00640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc510065180 0x7fc510067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.093+0000 7fc52be00640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc510065180 0x7fc510067630 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc53414d070 tx=0x7fc51c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:46.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.096+0000 7fc529600640 1 -- 172.21.15.16:0/3283744554 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc52404ff30 con 0x7fc5340c92e0 2024-08-17T02:45:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.280+0000 7fc539200640 1 -- 172.21.15.16:0/3283744554 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc534071050 con 0x7fc5340c92e0 2024-08-17T02:45:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.280+0000 7fc529600640 1 -- 172.21.15.16:0/3283744554 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc524053550 con 0x7fc5340c92e0 2024-08-17T02:45:46.282 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:45:46.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.283+0000 7fc539200640 1 -- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc510065180 msgr2=0x7fc510067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.283+0000 7fc539200640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc510065180 0x7fc510067630 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc53414d070 tx=0x7fc51c009040 comp rx=0 tx=0).stop 2024-08-17T02:45:46.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.283+0000 7fc539200640 1 -- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5340c92e0 msgr2=0x7fc5340ad270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.283+0000 7fc539200640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5340c92e0 0x7fc5340ad270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc52400c5b0 tx=0x7fc524007d70 comp rx=0 tx=0).stop 2024-08-17T02:45:46.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.283+0000 7fc539200640 1 -- 172.21.15.16:0/3283744554 shutdown_connections 2024-08-17T02:45:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.283+0000 7fc539200640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc510065180 0x7fc510067630 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.283+0000 7fc539200640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc5341590b0 0x7fc534149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.283+0000 7fc539200640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc534158ce0 0x7fc534147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.283+0000 7fc539200640 1 --2- 172.21.15.16:0/3283744554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5340c92e0 0x7fc5340ad270 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.283+0000 7fc539200640 1 -- 172.21.15.16:0/3283744554 >> 172.21.15.16:0/3283744554 conn(0x7fc534067750 msgr2=0x7fc5340aaab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.283+0000 7fc539200640 1 -- 172.21.15.16:0/3283744554 shutdown_connections 2024-08-17T02:45:46.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.283+0000 7fc539200640 1 -- 172.21.15.16:0/3283744554 wait complete. 2024-08-17T02:45:46.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:45:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.454+0000 7efe5dc00640 1 Processor -- start 2024-08-17T02:45:46.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.455+0000 7efe5dc00640 1 -- start start 2024-08-17T02:45:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.456+0000 7efe5dc00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe580ce250 0x7efe580ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.456+0000 7efe5dc00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe580c5a40 con 0x7efe580cb980 2024-08-17T02:45:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.456+0000 7efe5dc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe580c4e30 con 0x7efe580c4950 2024-08-17T02:45:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.456+0000 7efe5dc00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe580c5110 con 0x7efe580ce250 2024-08-17T02:45:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.456+0000 7efe57e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe580ce250 0x7efe580ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.456+0000 7efe5d200640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efe580cb980 0x7efe58158730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52780/0 (socket says 172.21.15.16:52780) 2024-08-17T02:45:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.456+0000 7efe5d200640 1 -- 172.21.15.16:0/2084639428 learned_addr learned my addr 172.21.15.16:0/2084639428 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.457+0000 7efe57e00640 1 -- 172.21.15.16:0/2084639428 >> v1:172.21.15.142:6789/0 conn(0x7efe580c4950 legacy=0x7efe580b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.457+0000 7efe57e00640 1 -- 172.21.15.16:0/2084639428 >> v1:172.21.15.16:6789/0 conn(0x7efe580cb980 legacy=0x7efe58158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.457+0000 7efe57e00640 1 -- 172.21.15.16:0/2084639428 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe580c53f0 con 0x7efe580ce250 2024-08-17T02:45:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.457+0000 7efe57400640 1 -- 172.21.15.16:0/2084639428 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654011162 0 0) 0x7efe580c5a40 con 0x7efe580cb980 2024-08-17T02:45:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.458+0000 7efe57e00640 1 --2- 172.21.15.16:0/2084639428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe580ce250 0x7efe580ce620 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efe48009850 tx=0x7efe480090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c52e6307bcbaa2 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.458+0000 7efe57400640 1 -- 172.21.15.16:0/2084639428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4802ae20 con 0x7efe580ce250 2024-08-17T02:45:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.458+0000 7efe57400640 1 -- 172.21.15.16:0/2084639428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe48032440 con 0x7efe580ce250 2024-08-17T02:45:46.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.458+0000 7efe57400640 1 -- 172.21.15.16:0/2084639428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe480327e0 con 0x7efe580ce250 2024-08-17T02:45:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.458+0000 7efe5dc00640 1 -- 172.21.15.16:0/2084639428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe580ce250 msgr2=0x7efe580ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.458+0000 7efe5dc00640 1 --2- 172.21.15.16:0/2084639428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe580ce250 0x7efe580ce620 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efe48009850 tx=0x7efe480090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.459+0000 7efe5dc00640 1 -- 172.21.15.16:0/2084639428 shutdown_connections 2024-08-17T02:45:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.459+0000 7efe5dc00640 1 --2- 172.21.15.16:0/2084639428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe580ce250 0x7efe580ce620 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.459+0000 7efe5dc00640 1 -- 172.21.15.16:0/2084639428 >> 172.21.15.16:0/2084639428 conn(0x7efe58098e30 msgr2=0x7efe5806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.459+0000 7efe5dc00640 1 -- 172.21.15.16:0/2084639428 shutdown_connections 2024-08-17T02:45:46.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.459+0000 7efe5dc00640 1 -- 172.21.15.16:0/2084639428 wait complete. 2024-08-17T02:45:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.460+0000 7efe5dc00640 1 Processor -- start 2024-08-17T02:45:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.460+0000 7efe5dc00640 1 -- start start 2024-08-17T02:45:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.461+0000 7efe5dc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe580c4950 0x7efe5813fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.461+0000 7efe5dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe580cb980 0x7efe581541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.461+0000 7efe5dc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe580ce250 0x7efe58156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.461+0000 7efe5dc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe580c6c60 con 0x7efe580cb980 2024-08-17T02:45:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.461+0000 7efe5dc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe580c7500 con 0x7efe580ce250 2024-08-17T02:45:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.461+0000 7efe5dc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe58140320 con 0x7efe580c4950 2024-08-17T02:45:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.461+0000 7efe4fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe580cb980 0x7efe581541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.461+0000 7efe4fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe580cb980 0x7efe581541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36388/0 (socket says 172.21.15.16:36388) 2024-08-17T02:45:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.461+0000 7efe4fe00640 1 -- 172.21.15.16:0/307517784 learned_addr learned my addr 172.21.15.16:0/307517784 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.461+0000 7efe57e00640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe580c4950 0x7efe5813fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.462+0000 7efe5d200640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe580ce250 0x7efe58156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.462+0000 7efe4fe00640 1 -- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe580c4950 msgr2=0x7efe5813fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.462+0000 7efe4fe00640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe580c4950 0x7efe5813fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.462+0000 7efe4fe00640 1 -- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe580ce250 msgr2=0x7efe58156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.462+0000 7efe4fe00640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe580ce250 0x7efe58156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.462+0000 7efe4fe00640 1 -- 172.21.15.16:0/307517784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe48009390 con 0x7efe580cb980 2024-08-17T02:45:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.462+0000 7efe4fe00640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe580cb980 0x7efe581541a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c590 tx=0x7efe40007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.462+0000 7efe4f400640 1 -- 172.21.15.16:0/307517784 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40012630 con 0x7efe580cb980 2024-08-17T02:45:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.462+0000 7efe4f400640 1 -- 172.21.15.16:0/307517784 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe400127b0 con 0x7efe580cb980 2024-08-17T02:45:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.462+0000 7efe5dc00640 1 -- 172.21.15.16:0/307517784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe58154c20 con 0x7efe580cb980 2024-08-17T02:45:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.462+0000 7efe4f400640 1 -- 172.21.15.16:0/307517784 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40012b70 con 0x7efe580cb980 2024-08-17T02:45:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.463+0000 7efe5dc00640 1 -- 172.21.15.16:0/307517784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe580c7190 con 0x7efe580cb980 2024-08-17T02:45:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.464+0000 7efe5dc00640 1 -- 172.21.15.16:0/307517784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe58071050 con 0x7efe580cb980 2024-08-17T02:45:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.464+0000 7efe4f400640 1 -- 172.21.15.16:0/307517784 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7efe4001b400 con 0x7efe580cb980 2024-08-17T02:45:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.465+0000 7efe4f400640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efe34065150 0x7efe34067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.465+0000 7efe4f400640 1 -- 172.21.15.16:0/307517784 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efe40081ee0 con 0x7efe580cb980 2024-08-17T02:45:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.466+0000 7efe57e00640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efe34065150 0x7efe34067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.467+0000 7efe57e00640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efe34065150 0x7efe34067600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efe48034000 tx=0x7efe48032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:46.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.469+0000 7efe4f400640 1 -- 172.21.15.16:0/307517784 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7efe4004ffb0 con 0x7efe580cb980 2024-08-17T02:45:46.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.649+0000 7efe5dc00640 1 -- 172.21.15.16:0/307517784 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe580c7640 con 0x7efe580cb980 2024-08-17T02:45:46.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.650+0000 7efe4f400640 1 -- 172.21.15.16:0/307517784 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe400535d0 con 0x7efe580cb980 2024-08-17T02:45:46.651 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-08-17T02:45:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.651+0000 7efe5dc00640 1 -- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efe34065150 msgr2=0x7efe34067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.651+0000 7efe5dc00640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efe34065150 0x7efe34067600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efe48034000 tx=0x7efe48032000 comp rx=0 tx=0).stop 2024-08-17T02:45:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.651+0000 7efe5dc00640 1 -- 172.21.15.16:0/307517784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe580cb980 msgr2=0x7efe581541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.651+0000 7efe5dc00640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe580cb980 0x7efe581541a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c590 tx=0x7efe40007ef0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.652+0000 7efe5dc00640 1 -- 172.21.15.16:0/307517784 shutdown_connections 2024-08-17T02:45:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.652+0000 7efe5dc00640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efe34065150 0x7efe34067600 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.652+0000 7efe5dc00640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe580ce250 0x7efe58156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.652+0000 7efe5dc00640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe580cb980 0x7efe581541a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.652+0000 7efe5dc00640 1 --2- 172.21.15.16:0/307517784 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7efe580c4950 0x7efe5813fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.652+0000 7efe5dc00640 1 -- 172.21.15.16:0/307517784 >> 172.21.15.16:0/307517784 conn(0x7efe58098e30 msgr2=0x7efe580c5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.652+0000 7efe5dc00640 1 -- 172.21.15.16:0/307517784 shutdown_connections 2024-08-17T02:45:46.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.652+0000 7efe5dc00640 1 -- 172.21.15.16:0/307517784 wait complete. 2024-08-17T02:45:46.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.825+0000 7f9ceae00640 1 Processor -- start 2024-08-17T02:45:46.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.825+0000 7f9ceae00640 1 -- start start 2024-08-17T02:45:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.827+0000 7f9ceae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ce406df00 0x7f9ce40adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.827+0000 7f9ceae00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9ce4155df0 0x7f9ce40ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.827+0000 7f9ceae00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce4156670 con 0x7f9ce4155a20 2024-08-17T02:45:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.827+0000 7f9ceae00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce40b0350 con 0x7f9ce4155df0 2024-08-17T02:45:46.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.827+0000 7f9ceae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce406e5e0 con 0x7f9ce406df00 2024-08-17T02:45:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.827+0000 7f9ce8e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9ce4155df0 0x7f9ce40ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.827+0000 7f9cdb400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ce406df00 0x7f9ce40adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.827+0000 7f9ce8e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9ce4155df0 0x7f9ce40ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:42696/0 (socket says 172.21.15.16:42696) 2024-08-17T02:45:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.827+0000 7f9ce8e00640 1 -- 172.21.15.16:0/587326622 learned_addr learned my addr 172.21.15.16:0/587326622 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.827+0000 7f9ce3e00640 1 --1- 172.21.15.16:0/587326622 >> v1:172.21.15.142:6789/0 conn(0x7f9ce4155a20 0x7f9ce406d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:39386/0 (socket says 172.21.15.16:39386) 2024-08-17T02:45:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.828+0000 7f9ce8e00640 1 -- 172.21.15.16:0/587326622 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ce406df00 msgr2=0x7f9ce40adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.828+0000 7f9ce8e00640 1 --2- 172.21.15.16:0/587326622 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ce406df00 0x7f9ce40adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.828+0000 7f9ce8e00640 1 -- 172.21.15.16:0/587326622 >> v1:172.21.15.142:6789/0 conn(0x7f9ce4155a20 legacy=0x7f9ce406d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.828+0000 7f9ce8e00640 1 -- 172.21.15.16:0/587326622 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce406e8c0 con 0x7f9ce4155df0 2024-08-17T02:45:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.829+0000 7f9ce8e00640 1 --2- 172.21.15.16:0/587326622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4155df0 0x7f9ce40ad9b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009f20 tx=0x7f9cdc009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7be8f19c04fa7270 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.829+0000 7f9ce3400640 1 -- 172.21.15.16:0/587326622 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc031400 con 0x7f9ce4155df0 2024-08-17T02:45:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.829+0000 7f9ce3400640 1 -- 172.21.15.16:0/587326622 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc02d050 con 0x7f9ce4155df0 2024-08-17T02:45:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.829+0000 7f9ce3400640 1 -- 172.21.15.16:0/587326622 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0317d0 con 0x7f9ce4155df0 2024-08-17T02:45:46.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.830+0000 7f9ceae00640 1 -- 172.21.15.16:0/587326622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4155df0 msgr2=0x7f9ce40ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.830+0000 7f9ceae00640 1 --2- 172.21.15.16:0/587326622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4155df0 0x7f9ce40ad9b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009f20 tx=0x7f9cdc009840 comp rx=0 tx=0).stop 2024-08-17T02:45:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.830+0000 7f9ceae00640 1 -- 172.21.15.16:0/587326622 shutdown_connections 2024-08-17T02:45:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.830+0000 7f9ceae00640 1 --2- 172.21.15.16:0/587326622 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ce406df00 0x7f9ce40adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.830+0000 7f9ceae00640 1 --2- 172.21.15.16:0/587326622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4155df0 0x7f9ce40ad9b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.830+0000 7f9ceae00640 1 -- 172.21.15.16:0/587326622 >> 172.21.15.16:0/587326622 conn(0x7f9ce40cb9b0 msgr2=0x7f9ce40cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.830+0000 7f9ceae00640 1 -- 172.21.15.16:0/587326622 shutdown_connections 2024-08-17T02:45:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.830+0000 7f9ceae00640 1 -- 172.21.15.16:0/587326622 wait complete. 2024-08-17T02:45:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.831+0000 7f9ceae00640 1 Processor -- start 2024-08-17T02:45:46.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.831+0000 7f9ceae00640 1 -- start start 2024-08-17T02:45:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.832+0000 7f9ceae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ce406df00 0x7f9ce414b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.832+0000 7f9ceae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce4155a20 0x7f9ce4154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.832+0000 7f9ceae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4155df0 0x7f9ce4154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.832+0000 7f9ceae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce414f410 con 0x7f9ce4155a20 2024-08-17T02:45:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.832+0000 7f9ceae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce414f6d0 con 0x7f9ce4155df0 2024-08-17T02:45:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.832+0000 7f9ceae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce414f9b0 con 0x7f9ce406df00 2024-08-17T02:45:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.833+0000 7f9cdb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce4155a20 0x7f9ce4154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.833+0000 7f9cdb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce4155a20 0x7f9ce4154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36400/0 (socket says 172.21.15.16:36400) 2024-08-17T02:45:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.833+0000 7f9cdb400640 1 -- 172.21.15.16:0/1871652228 learned_addr learned my addr 172.21.15.16:0/1871652228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:46.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.833+0000 7f9ce3e00640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ce406df00 0x7f9ce414b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.833+0000 7f9ce8e00640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4155df0 0x7f9ce4154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.833+0000 7f9cdb400640 1 -- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ce406df00 msgr2=0x7f9ce414b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.833+0000 7f9cdb400640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ce406df00 0x7f9ce414b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.833+0000 7f9cdb400640 1 -- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4155df0 msgr2=0x7f9ce4154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.833+0000 7f9cdb400640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4155df0 0x7f9ce4154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.833+0000 7f9cdb400640 1 -- 172.21.15.16:0/1871652228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc009ae0 con 0x7f9ce4155a20 2024-08-17T02:45:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.833+0000 7f9cdb400640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce4155a20 0x7f9ce4154500 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00ea20 tx=0x7f9ccc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.834+0000 7f9cdbe00640 1 -- 172.21.15.16:0/1871652228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc00ca10 con 0x7f9ce4155a20 2024-08-17T02:45:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.834+0000 7f9cdbe00640 1 -- 172.21.15.16:0/1871652228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ccc00cb70 con 0x7f9ce4155a20 2024-08-17T02:45:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.834+0000 7f9cdbe00640 1 -- 172.21.15.16:0/1871652228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc013470 con 0x7f9ce4155a20 2024-08-17T02:45:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.834+0000 7f9ceae00640 1 -- 172.21.15.16:0/1871652228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce414fc90 con 0x7f9ce4155a20 2024-08-17T02:45:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.834+0000 7f9ceae00640 1 -- 172.21.15.16:0/1871652228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce414ba40 con 0x7f9ce4155a20 2024-08-17T02:45:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.835+0000 7f9ceae00640 1 -- 172.21.15.16:0/1871652228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce4064db0 con 0x7f9ce4155a20 2024-08-17T02:45:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.836+0000 7f9cdbe00640 1 -- 172.21.15.16:0/1871652228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f9ccc013860 con 0x7f9ce4155a20 2024-08-17T02:45:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.836+0000 7f9cdbe00640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9cc0065150 0x7f9cc0067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.837+0000 7f9cdbe00640 1 -- 172.21.15.16:0/1871652228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9ccc082880 con 0x7f9ce4155a20 2024-08-17T02:45:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.837+0000 7f9ce3e00640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9cc0065150 0x7f9cc0067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.838+0000 7f9ce3e00640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9cc0065150 0x7f9cc0067600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4007790 tx=0x7f9cd4007ce0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:46.841+0000 7f9cdbe00640 1 -- 172.21.15.16:0/1871652228 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9ccc0508a0 con 0x7f9ce4155a20 2024-08-17T02:45:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.024+0000 7f9ceae00640 1 -- 172.21.15.16:0/1871652228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ce4159300 con 0x7f9ce4155a20 2024-08-17T02:45:47.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.025+0000 7f9cdbe00640 1 -- 172.21.15.16:0/1871652228 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f9ccc053ec0 con 0x7f9ce4155a20 2024-08-17T02:45:47.027 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:47.027 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.028+0000 7f9ceae00640 1 -- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9cc0065150 msgr2=0x7f9cc0067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.028+0000 7f9ceae00640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9cc0065150 0x7f9cc0067600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4007790 tx=0x7f9cd4007ce0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.028+0000 7f9ceae00640 1 -- 172.21.15.16:0/1871652228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce4155a20 msgr2=0x7f9ce4154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.028+0000 7f9ceae00640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce4155a20 0x7f9ce4154500 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00ea20 tx=0x7f9ccc0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.028+0000 7f9ceae00640 1 -- 172.21.15.16:0/1871652228 shutdown_connections 2024-08-17T02:45:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.028+0000 7f9ceae00640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9cc0065150 0x7f9cc0067600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.028+0000 7f9ceae00640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ce4155df0 0x7f9ce4154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.028+0000 7f9ceae00640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ce4155a20 0x7f9ce4154500 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.028+0000 7f9ceae00640 1 --2- 172.21.15.16:0/1871652228 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9ce406df00 0x7f9ce414b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.028+0000 7f9ceae00640 1 -- 172.21.15.16:0/1871652228 >> 172.21.15.16:0/1871652228 conn(0x7f9ce40cb9b0 msgr2=0x7f9ce406c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.028+0000 7f9ceae00640 1 -- 172.21.15.16:0/1871652228 shutdown_connections 2024-08-17T02:45:47.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.028+0000 7f9ceae00640 1 -- 172.21.15.16:0/1871652228 wait complete. 2024-08-17T02:45:47.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:45:14.291903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:45:18.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:45:18.430718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:45:47.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:45:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.197+0000 7f78be200640 1 Processor -- start 2024-08-17T02:45:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.197+0000 7f78be200640 1 -- start start 2024-08-17T02:45:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.198+0000 7f78be200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b8158ed0 0x7f78b8069a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.198+0000 7f78be200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78b81592a0 0x7f78b80abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.198+0000 7f78be200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f78b8096a70 0x7f78b8096eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.198+0000 7f78be200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b80b10e0 con 0x7f78b81592a0 2024-08-17T02:45:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.198+0000 7f78be200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b80973f0 con 0x7f78b8096a70 2024-08-17T02:45:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.198+0000 7f78be200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b80ac3f0 con 0x7f78b8158ed0 2024-08-17T02:45:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.198+0000 7f78afe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78b81592a0 0x7f78b80abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.198+0000 7f78b7400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b8158ed0 0x7f78b8069a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.198+0000 7f78afe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78b81592a0 0x7f78b80abeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36416/0 (socket says 172.21.15.16:36416) 2024-08-17T02:45:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.198+0000 7f78afe00640 1 -- 172.21.15.16:0/3120104337 learned_addr learned my addr 172.21.15.16:0/3120104337 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.198+0000 7f78b7e00640 1 --2- 172.21.15.16:0/3120104337 >> v2:172.21.15.142:3300/0 conn(0x7f78b8096a70 0x7f78b8096eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.199+0000 7f78afe00640 1 -- 172.21.15.16:0/3120104337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b8158ed0 msgr2=0x7f78b8069a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.199+0000 7f78afe00640 1 --2- 172.21.15.16:0/3120104337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b8158ed0 0x7f78b8069a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.199+0000 7f78afe00640 1 -- 172.21.15.16:0/3120104337 >> v2:172.21.15.142:3300/0 conn(0x7f78b8096a70 msgr2=0x7f78b8096eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.199+0000 7f78afe00640 1 --2- 172.21.15.16:0/3120104337 >> v2:172.21.15.142:3300/0 conn(0x7f78b8096a70 0x7f78b8096eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.199+0000 7f78afe00640 1 -- 172.21.15.16:0/3120104337 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b80ac680 con 0x7f78b81592a0 2024-08-17T02:45:47.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.199+0000 7f78b7e00640 1 --2- 172.21.15.16:0/3120104337 >> v2:172.21.15.142:3300/0 conn(0x7f78b8096a70 0x7f78b8096eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:47.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.200+0000 7f78afe00640 1 --2- 172.21.15.16:0/3120104337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b81592a0 0x7f78b80abeb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f78b80acdb0 tx=0x7f78a00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9adac6fcb749826 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:47.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.200+0000 7f78b6a00640 1 -- 172.21.15.16:0/3120104337 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a002add0 con 0x7f78b81592a0 2024-08-17T02:45:47.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.200+0000 7f78b6a00640 1 -- 172.21.15.16:0/3120104337 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a0032440 con 0x7f78b81592a0 2024-08-17T02:45:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.200+0000 7f78b6a00640 1 -- 172.21.15.16:0/3120104337 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0032820 con 0x7f78b81592a0 2024-08-17T02:45:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.201+0000 7f78be200640 1 -- 172.21.15.16:0/3120104337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b81592a0 msgr2=0x7f78b80abeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.201+0000 7f78be200640 1 --2- 172.21.15.16:0/3120104337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b81592a0 0x7f78b80abeb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f78b80acdb0 tx=0x7f78a00091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.201+0000 7f78be200640 1 -- 172.21.15.16:0/3120104337 shutdown_connections 2024-08-17T02:45:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.201+0000 7f78be200640 1 --2- 172.21.15.16:0/3120104337 >> v2:172.21.15.142:3300/0 conn(0x7f78b8096a70 0x7f78b8096eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.201+0000 7f78be200640 1 --2- 172.21.15.16:0/3120104337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b81592a0 0x7f78b80abeb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.201+0000 7f78be200640 1 --2- 172.21.15.16:0/3120104337 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b8158ed0 0x7f78b8069a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.201+0000 7f78be200640 1 -- 172.21.15.16:0/3120104337 >> 172.21.15.16:0/3120104337 conn(0x7f78b80b30b0 msgr2=0x7f78b80b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.201+0000 7f78be200640 1 -- 172.21.15.16:0/3120104337 shutdown_connections 2024-08-17T02:45:47.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.201+0000 7f78be200640 1 -- 172.21.15.16:0/3120104337 wait complete. 2024-08-17T02:45:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.202+0000 7f78be200640 1 Processor -- start 2024-08-17T02:45:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.202+0000 7f78be200640 1 -- start start 2024-08-17T02:45:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.203+0000 7f78be200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78b8096a70 0x7f78b80ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.203+0000 7f78be200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b8158ed0 0x7f78b8155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.203+0000 7f78be200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b8155660 0x7f78b80ad340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.203+0000 7f78be200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b8099140 con 0x7f78b8158ed0 2024-08-17T02:45:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.203+0000 7f78be200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b809ab00 con 0x7f78b8096a70 2024-08-17T02:45:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.203+0000 7f78be200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b809a510 con 0x7f78b8155660 2024-08-17T02:45:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.203+0000 7f78afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b8158ed0 0x7f78b8155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.203+0000 7f78afe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b8158ed0 0x7f78b8155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36420/0 (socket says 172.21.15.16:36420) 2024-08-17T02:45:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.203+0000 7f78afe00640 1 -- 172.21.15.16:0/3809081232 learned_addr learned my addr 172.21.15.16:0/3809081232 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78b7400640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78b8096a70 0x7f78b80ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78b7e00640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b8155660 0x7f78b80ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78afe00640 1 -- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b8155660 msgr2=0x7f78b80ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78afe00640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b8155660 0x7f78b80ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78afe00640 1 -- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78b8096a70 msgr2=0x7f78b80ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78afe00640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78b8096a70 0x7f78b80ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78afe00640 1 -- 172.21.15.16:0/3809081232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0009450 con 0x7f78b8158ed0 2024-08-17T02:45:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78b7400640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78b8096a70 0x7f78b80ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:47.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78afe00640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b8158ed0 0x7f78b8155120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009210 tx=0x7f78a0032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:47.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78aea00640 1 -- 172.21.15.16:0/3809081232 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0000c40 con 0x7f78b8158ed0 2024-08-17T02:45:47.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78aea00640 1 -- 172.21.15.16:0/3809081232 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a0031a60 con 0x7f78b8158ed0 2024-08-17T02:45:47.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78aea00640 1 -- 172.21.15.16:0/3809081232 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0031e60 con 0x7f78b8158ed0 2024-08-17T02:45:47.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.204+0000 7f78be200640 1 -- 172.21.15.16:0/3809081232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b80add20 con 0x7f78b8158ed0 2024-08-17T02:45:47.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.205+0000 7f78be200640 1 -- 172.21.15.16:0/3809081232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b809a790 con 0x7f78b8158ed0 2024-08-17T02:45:47.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.205+0000 7f78be200640 1 -- 172.21.15.16:0/3809081232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b809acb0 con 0x7f78b8158ed0 2024-08-17T02:45:47.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.206+0000 7f78aea00640 1 -- 172.21.15.16:0/3809081232 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f78a0039040 con 0x7f78b8158ed0 2024-08-17T02:45:47.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.207+0000 7f78aea00640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7894065150 0x7f7894067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.207+0000 7f78aea00640 1 -- 172.21.15.16:0/3809081232 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f78a00a2100 con 0x7f78b8158ed0 2024-08-17T02:45:47.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.207+0000 7f78b7400640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7894065150 0x7f7894067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.208+0000 7f78b7400640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7894065150 0x7f7894067600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009c50 tx=0x7f78a8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:47.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.211+0000 7f78aea00640 1 -- 172.21.15.16:0/3809081232 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f78b809acb0 con 0x7f78b8158ed0 2024-08-17T02:45:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.392+0000 7f78be200640 1 -- 172.21.15.16:0/3809081232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78b809acb0 con 0x7f78b8158ed0 2024-08-17T02:45:47.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.392+0000 7f78aea00640 1 -- 172.21.15.16:0/3809081232 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78b809acb0 con 0x7f78b8158ed0 2024-08-17T02:45:47.394 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-17T02:45:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.394+0000 7f78be200640 1 -- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7894065150 msgr2=0x7f7894067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.394+0000 7f78be200640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7894065150 0x7f7894067600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009c50 tx=0x7f78a8008040 comp rx=0 tx=0).stop 2024-08-17T02:45:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.394+0000 7f78be200640 1 -- 172.21.15.16:0/3809081232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b8158ed0 msgr2=0x7f78b8155120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.394+0000 7f78be200640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b8158ed0 0x7f78b8155120 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009210 tx=0x7f78a0032000 comp rx=0 tx=0).stop 2024-08-17T02:45:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.394+0000 7f78be200640 1 -- 172.21.15.16:0/3809081232 shutdown_connections 2024-08-17T02:45:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.394+0000 7f78be200640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7894065150 0x7f7894067600 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.394+0000 7f78be200640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f78b8155660 0x7f78b80ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.394+0000 7f78be200640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78b8158ed0 0x7f78b8155120 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.394+0000 7f78be200640 1 --2- 172.21.15.16:0/3809081232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f78b8096a70 0x7f78b80ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.394+0000 7f78be200640 1 -- 172.21.15.16:0/3809081232 >> 172.21.15.16:0/3809081232 conn(0x7f78b80b30b0 msgr2=0x7f78b80b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:47.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.394+0000 7f78be200640 1 -- 172.21.15.16:0/3809081232 shutdown_connections 2024-08-17T02:45:47.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.395+0000 7f78be200640 1 -- 172.21.15.16:0/3809081232 wait complete. 2024-08-17T02:45:47.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:45:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.563+0000 7f70db000640 1 Processor -- start 2024-08-17T02:45:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.563+0000 7f70db000640 1 -- start start 2024-08-17T02:45:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.564+0000 7f70db000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70d4156c50 0x7f70d40c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.564+0000 7f70db000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f70d40adcf0 0x7f70d40ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.564+0000 7f70db000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70d40abb00 0x7f70d40c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.564+0000 7f70db000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d40ab130 con 0x7f70d40abb00 2024-08-17T02:45:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.564+0000 7f70db000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d40ae5a0 con 0x7f70d40adcf0 2024-08-17T02:45:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.564+0000 7f70db000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d40ae2c0 con 0x7f70d4156c50 2024-08-17T02:45:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.564+0000 7f70d9000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70d40abb00 0x7f70d40c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.564+0000 7f70d9000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70d40abb00 0x7f70d40c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36434/0 (socket says 172.21.15.16:36434) 2024-08-17T02:45:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.564+0000 7f70d9000640 1 -- 172.21.15.16:0/1039202397 learned_addr learned my addr 172.21.15.16:0/1039202397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.564+0000 7f70d3e00640 1 --2- 172.21.15.16:0/1039202397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70d4156c50 0x7f70d40c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.565+0000 7f70cbe00640 1 --2- 172.21.15.16:0/1039202397 >> v2:172.21.15.142:3300/0 conn(0x7f70d40adcf0 0x7f70d40ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.565+0000 7f70d9000640 1 -- 172.21.15.16:0/1039202397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70d4156c50 msgr2=0x7f70d40c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.565+0000 7f70d9000640 1 --2- 172.21.15.16:0/1039202397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70d4156c50 0x7f70d40c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.565+0000 7f70d9000640 1 -- 172.21.15.16:0/1039202397 >> v2:172.21.15.142:3300/0 conn(0x7f70d40adcf0 msgr2=0x7f70d40ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.565+0000 7f70d9000640 1 --2- 172.21.15.16:0/1039202397 >> v2:172.21.15.142:3300/0 conn(0x7f70d40adcf0 0x7f70d40ab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.565+0000 7f70d9000640 1 -- 172.21.15.16:0/1039202397 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d40aaae0 con 0x7f70d40abb00 2024-08-17T02:45:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.565+0000 7f70cbe00640 1 --2- 172.21.15.16:0/1039202397 >> v2:172.21.15.142:3300/0 conn(0x7f70d40adcf0 0x7f70d40ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.566+0000 7f70d9000640 1 --2- 172.21.15.16:0/1039202397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d40abb00 0x7f70d40c8bd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f70d406bc10 tx=0x7f70cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe0e5c18c978217 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.566+0000 7f70d3400640 1 -- 172.21.15.16:0/1039202397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc031400 con 0x7f70d40abb00 2024-08-17T02:45:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.566+0000 7f70d3400640 1 -- 172.21.15.16:0/1039202397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70cc02d050 con 0x7f70d40abb00 2024-08-17T02:45:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.566+0000 7f70d3400640 1 -- 172.21.15.16:0/1039202397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc0317d0 con 0x7f70d40abb00 2024-08-17T02:45:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.567+0000 7f70db000640 1 -- 172.21.15.16:0/1039202397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d40abb00 msgr2=0x7f70d40c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.567+0000 7f70db000640 1 --2- 172.21.15.16:0/1039202397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d40abb00 0x7f70d40c8bd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f70d406bc10 tx=0x7f70cc009840 comp rx=0 tx=0).stop 2024-08-17T02:45:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.567+0000 7f70db000640 1 -- 172.21.15.16:0/1039202397 shutdown_connections 2024-08-17T02:45:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.567+0000 7f70db000640 1 --2- 172.21.15.16:0/1039202397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d40abb00 0x7f70d40c8bd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.567+0000 7f70db000640 1 --2- 172.21.15.16:0/1039202397 >> v2:172.21.15.142:3300/0 conn(0x7f70d40adcf0 0x7f70d40ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.567+0000 7f70db000640 1 --2- 172.21.15.16:0/1039202397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70d4156c50 0x7f70d40c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.567+0000 7f70db000640 1 -- 172.21.15.16:0/1039202397 >> 172.21.15.16:0/1039202397 conn(0x7f70d4067750 msgr2=0x7f70d40c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.567+0000 7f70db000640 1 -- 172.21.15.16:0/1039202397 shutdown_connections 2024-08-17T02:45:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.567+0000 7f70db000640 1 -- 172.21.15.16:0/1039202397 wait complete. 2024-08-17T02:45:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.568+0000 7f70db000640 1 Processor -- start 2024-08-17T02:45:47.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.568+0000 7f70db000640 1 -- start start 2024-08-17T02:45:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.569+0000 7f70db000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70d40abb00 0x7f70d416cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.569+0000 7f70db000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d40adcf0 0x7f70d40a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.569+0000 7f70db000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70d4156c50 0x7f70d415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.569+0000 7f70db000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d415d1c0 con 0x7f70d40adcf0 2024-08-17T02:45:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.569+0000 7f70db000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d415d4a0 con 0x7f70d4156c50 2024-08-17T02:45:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.569+0000 7f70db000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d416c4a0 con 0x7f70d40abb00 2024-08-17T02:45:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.569+0000 7f70cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d40adcf0 0x7f70d40a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.569+0000 7f70cbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d40adcf0 0x7f70d40a9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36438/0 (socket says 172.21.15.16:36438) 2024-08-17T02:45:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.569+0000 7f70cbe00640 1 -- 172.21.15.16:0/3124551371 learned_addr learned my addr 172.21.15.16:0/3124551371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70d3e00640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70d40abb00 0x7f70d416cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70d9000640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70d4156c50 0x7f70d415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70cbe00640 1 -- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70d40abb00 msgr2=0x7f70d416cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70cbe00640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70d40abb00 0x7f70d416cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70cbe00640 1 -- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70d4156c50 msgr2=0x7f70d415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70cbe00640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70d4156c50 0x7f70d415c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70cbe00640 1 -- 172.21.15.16:0/3124551371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc009ae0 con 0x7f70d40adcf0 2024-08-17T02:45:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70d9000640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70d4156c50 0x7f70d415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70cbe00640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d40adcf0 0x7f70d40a9b10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00d890 tx=0x7f70bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70cb400640 1 -- 172.21.15.16:0/3124551371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc00de80 con 0x7f70d40adcf0 2024-08-17T02:45:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70cb400640 1 -- 172.21.15.16:0/3124551371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70bc00b8d0 con 0x7f70d40adcf0 2024-08-17T02:45:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70cb400640 1 -- 172.21.15.16:0/3124551371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc00bc90 con 0x7f70d40adcf0 2024-08-17T02:45:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.570+0000 7f70db000640 1 -- 172.21.15.16:0/3124551371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d416c780 con 0x7f70d40adcf0 2024-08-17T02:45:47.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.571+0000 7f70db000640 1 -- 172.21.15.16:0/3124551371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d40aa050 con 0x7f70d40adcf0 2024-08-17T02:45:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.572+0000 7f70db000640 1 -- 172.21.15.16:0/3124551371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d416cc00 con 0x7f70d40adcf0 2024-08-17T02:45:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.572+0000 7f70cb400640 1 -- 172.21.15.16:0/3124551371 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f70bc01e460 con 0x7f70d40adcf0 2024-08-17T02:45:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.573+0000 7f70cb400640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f70b0065150 0x7f70b0067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.573+0000 7f70cb400640 1 -- 172.21.15.16:0/3124551371 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f70bc082b60 con 0x7f70d40adcf0 2024-08-17T02:45:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.574+0000 7f70d3e00640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f70b0065150 0x7f70b0067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.574+0000 7f70d3e00640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f70b0065150 0x7f70b0067600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009c80 tx=0x7f70c40090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:47.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.577+0000 7f70cb400640 1 -- 172.21.15.16:0/3124551371 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f70bc050b80 con 0x7f70d40adcf0 2024-08-17T02:45:47.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.757+0000 7f70db000640 1 -- 172.21.15.16:0/3124551371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70d40aa240 con 0x7f70d40adcf0 2024-08-17T02:45:47.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.757+0000 7f70cb400640 1 -- 172.21.15.16:0/3124551371 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f70bc0541a0 con 0x7f70d40adcf0 2024-08-17T02:45:47.759 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:45:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.759+0000 7f70db000640 1 -- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f70b0065150 msgr2=0x7f70b0067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.759+0000 7f70db000640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f70b0065150 0x7f70b0067600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009c80 tx=0x7f70c40090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.759+0000 7f70db000640 1 -- 172.21.15.16:0/3124551371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d40adcf0 msgr2=0x7f70d40a9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.759+0000 7f70db000640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d40adcf0 0x7f70d40a9b10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00d890 tx=0x7f70bc008040 comp rx=0 tx=0).stop 2024-08-17T02:45:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.759+0000 7f70db000640 1 -- 172.21.15.16:0/3124551371 shutdown_connections 2024-08-17T02:45:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.759+0000 7f70db000640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f70b0065150 0x7f70b0067600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.759+0000 7f70db000640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70d4156c50 0x7f70d415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.759+0000 7f70db000640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d40adcf0 0x7f70d40a9b10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.759+0000 7f70db000640 1 --2- 172.21.15.16:0/3124551371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f70d40abb00 0x7f70d416cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.759+0000 7f70db000640 1 -- 172.21.15.16:0/3124551371 >> 172.21.15.16:0/3124551371 conn(0x7f70d4067750 msgr2=0x7f70d4155eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.759+0000 7f70db000640 1 -- 172.21.15.16:0/3124551371 shutdown_connections 2024-08-17T02:45:47.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.759+0000 7f70db000640 1 -- 172.21.15.16:0/3124551371 wait complete. 2024-08-17T02:45:47.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:45:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.930+0000 7f4f2d600640 1 Processor -- start 2024-08-17T02:45:47.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.930+0000 7f4f2d600640 1 -- start start 2024-08-17T02:45:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.931+0000 7f4f2d600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f28158da0 0x7f4f280ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.931+0000 7f4f2d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f28159170 0x7f4f280aa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.931+0000 7f4f2d600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f280681f0 con 0x7f4f280aad50 2024-08-17T02:45:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.931+0000 7f4f2d600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f280ab1d0 con 0x7f4f28158da0 2024-08-17T02:45:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.931+0000 7f4f2d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f280b08c0 con 0x7f4f28159170 2024-08-17T02:45:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.932+0000 7f4f26a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f28158da0 0x7f4f280ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.932+0000 7f4f27400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4f280aad50 0x7f4f28068680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52794/0 (socket says 172.21.15.16:52794) 2024-08-17T02:45:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.932+0000 7f4f27400640 1 -- 172.21.15.16:0/2189170957 learned_addr learned my addr 172.21.15.16:0/2189170957 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.932+0000 7f4f1fe00640 1 --2- 172.21.15.16:0/2189170957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f28159170 0x7f4f280aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.932+0000 7f4f26a00640 1 -- 172.21.15.16:0/2189170957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f28159170 msgr2=0x7f4f280aa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.932+0000 7f4f26a00640 1 --2- 172.21.15.16:0/2189170957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f28159170 0x7f4f280aa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.932+0000 7f4f26a00640 1 -- 172.21.15.16:0/2189170957 >> v1:172.21.15.16:6789/0 conn(0x7f4f280aad50 legacy=0x7f4f28068680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.932+0000 7f4f26a00640 1 -- 172.21.15.16:0/2189170957 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f280aca00 con 0x7f4f28158da0 2024-08-17T02:45:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.932+0000 7f4f26000640 1 -- 172.21.15.16:0/2189170957 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476757232 0 0) 0x7f4f280681f0 con 0x7f4f280aad50 2024-08-17T02:45:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.933+0000 7f4f26a00640 1 --2- 172.21.15.16:0/2189170957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f28158da0 0x7f4f280ac3e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009850 tx=0x7f4f180090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8e44e3c461fa6b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.933+0000 7f4f26000640 1 -- 172.21.15.16:0/2189170957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1802ad00 con 0x7f4f28158da0 2024-08-17T02:45:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.933+0000 7f4f26000640 1 -- 172.21.15.16:0/2189170957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1802ae60 con 0x7f4f28158da0 2024-08-17T02:45:47.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.933+0000 7f4f26000640 1 -- 172.21.15.16:0/2189170957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18032670 con 0x7f4f28158da0 2024-08-17T02:45:47.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.934+0000 7f4f2d600640 1 -- 172.21.15.16:0/2189170957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f28158da0 msgr2=0x7f4f280ac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.934+0000 7f4f2d600640 1 --2- 172.21.15.16:0/2189170957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f28158da0 0x7f4f280ac3e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009850 tx=0x7f4f180090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.934+0000 7f4f2d600640 1 -- 172.21.15.16:0/2189170957 shutdown_connections 2024-08-17T02:45:47.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.934+0000 7f4f2d600640 1 --2- 172.21.15.16:0/2189170957 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f28159170 0x7f4f280aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.934+0000 7f4f2d600640 1 --2- 172.21.15.16:0/2189170957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f28158da0 0x7f4f280ac3e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.934+0000 7f4f2d600640 1 -- 172.21.15.16:0/2189170957 >> 172.21.15.16:0/2189170957 conn(0x7f4f28096a70 msgr2=0x7f4f28096e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:47.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.934+0000 7f4f2d600640 1 -- 172.21.15.16:0/2189170957 shutdown_connections 2024-08-17T02:45:47.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.934+0000 7f4f2d600640 1 -- 172.21.15.16:0/2189170957 wait complete. 2024-08-17T02:45:47.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.935+0000 7f4f2d600640 1 Processor -- start 2024-08-17T02:45:47.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.936+0000 7f4f2d600640 1 -- start start 2024-08-17T02:45:47.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.936+0000 7f4f2d600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f280aad50 0x7f4f28154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.936+0000 7f4f2d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f28158da0 0x7f4f280b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.936+0000 7f4f2d600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f280b3770 0x7f4f280b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.936+0000 7f4f2d600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2815b220 con 0x7f4f280b3770 2024-08-17T02:45:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.936+0000 7f4f2d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28154ca0 con 0x7f4f28158da0 2024-08-17T02:45:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.936+0000 7f4f2d600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2815a610 con 0x7f4f280aad50 2024-08-17T02:45:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.937+0000 7f4f27400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f280b3770 0x7f4f280b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.937+0000 7f4f27400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f280b3770 0x7f4f280b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36456/0 (socket says 172.21.15.16:36456) 2024-08-17T02:45:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.937+0000 7f4f27400640 1 -- 172.21.15.16:0/2082875311 learned_addr learned my addr 172.21.15.16:0/2082875311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.937+0000 7f4f26a00640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f280aad50 0x7f4f28154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.937+0000 7f4f1fe00640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f28158da0 0x7f4f280b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.937+0000 7f4f27400640 1 -- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f280aad50 msgr2=0x7f4f28154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.937+0000 7f4f27400640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f280aad50 0x7f4f28154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.937+0000 7f4f27400640 1 -- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f28158da0 msgr2=0x7f4f280b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.937+0000 7f4f27400640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f28158da0 0x7f4f280b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.937+0000 7f4f27400640 1 -- 172.21.15.16:0/2082875311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f18009390 con 0x7f4f280b3770 2024-08-17T02:45:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.938+0000 7f4f27400640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f280b3770 0x7f4f280b1540 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000cc40 tx=0x7f4f20001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.938+0000 7f4f1e000640 1 -- 172.21.15.16:0/2082875311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2000ce60 con 0x7f4f280b3770 2024-08-17T02:45:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.938+0000 7f4f1e000640 1 -- 172.21.15.16:0/2082875311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2000a7f0 con 0x7f4f280b3770 2024-08-17T02:45:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.938+0000 7f4f1e000640 1 -- 172.21.15.16:0/2082875311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2000ab90 con 0x7f4f280b3770 2024-08-17T02:45:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.938+0000 7f4f2d600640 1 -- 172.21.15.16:0/2082875311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2815a8f0 con 0x7f4f280b3770 2024-08-17T02:45:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.938+0000 7f4f2d600640 1 -- 172.21.15.16:0/2082875311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2815aeb0 con 0x7f4f280b3770 2024-08-17T02:45:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.939+0000 7f4f2d600640 1 -- 172.21.15.16:0/2082875311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2815b3d0 con 0x7f4f280b3770 2024-08-17T02:45:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.940+0000 7f4f1e000640 1 -- 172.21.15.16:0/2082875311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f4f200070f0 con 0x7f4f280b3770 2024-08-17T02:45:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.941+0000 7f4f1e000640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4f04065150 0x7f4f04067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.941+0000 7f4f1e000640 1 -- 172.21.15.16:0/2082875311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4f200825d0 con 0x7f4f280b3770 2024-08-17T02:45:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.941+0000 7f4f26a00640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4f04065150 0x7f4f04067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.942+0000 7f4f26a00640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4f04065150 0x7f4f04067600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f18034000 tx=0x7f4f18032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:47.945+0000 7f4f1e000640 1 -- 172.21.15.16:0/2082875311 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4f20050750 con 0x7f4f280b3770 2024-08-17T02:45:48.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.127+0000 7f4f2d600640 1 -- 172.21.15.16:0/2082875311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4f28071050 con 0x7f4f280b3770 2024-08-17T02:45:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.127+0000 7f4f1e000640 1 -- 172.21.15.16:0/2082875311 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4f20053d70 con 0x7f4f280b3770 2024-08-17T02:45:48.129 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-08-17T02:45:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.129+0000 7f4f2d600640 1 -- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4f04065150 msgr2=0x7f4f04067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.129+0000 7f4f2d600640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4f04065150 0x7f4f04067600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f18034000 tx=0x7f4f18032000 comp rx=0 tx=0).stop 2024-08-17T02:45:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.129+0000 7f4f2d600640 1 -- 172.21.15.16:0/2082875311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f280b3770 msgr2=0x7f4f280b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.129+0000 7f4f2d600640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f280b3770 0x7f4f280b1540 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000cc40 tx=0x7f4f20001970 comp rx=0 tx=0).stop 2024-08-17T02:45:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.129+0000 7f4f2d600640 1 -- 172.21.15.16:0/2082875311 shutdown_connections 2024-08-17T02:45:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.129+0000 7f4f2d600640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4f04065150 0x7f4f04067600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.129+0000 7f4f2d600640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f280b3770 0x7f4f280b1540 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.129+0000 7f4f2d600640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f28158da0 0x7f4f280b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.129+0000 7f4f2d600640 1 --2- 172.21.15.16:0/2082875311 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4f280aad50 0x7f4f28154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.129+0000 7f4f2d600640 1 -- 172.21.15.16:0/2082875311 >> 172.21.15.16:0/2082875311 conn(0x7f4f28096a70 msgr2=0x7f4f2806c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.129+0000 7f4f2d600640 1 -- 172.21.15.16:0/2082875311 shutdown_connections 2024-08-17T02:45:48.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.129+0000 7f4f2d600640 1 -- 172.21.15.16:0/2082875311 wait complete. 2024-08-17T02:45:48.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.296+0000 7f29d9c00640 1 Processor -- start 2024-08-17T02:45:48.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.296+0000 7f29d9c00640 1 -- start start 2024-08-17T02:45:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.297+0000 7f29d9c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d406df00 0x7f29d40adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.297+0000 7f29d9c00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d4156670 con 0x7f29d4155a20 2024-08-17T02:45:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29d9c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d406e5e0 con 0x7f29d4155df0 2024-08-17T02:45:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29d9c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40b0350 con 0x7f29d406df00 2024-08-17T02:45:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29d2a00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f29d4155a20 0x7f29d406d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52804/0 (socket says 172.21.15.16:52804) 2024-08-17T02:45:48.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29d2a00640 1 -- 172.21.15.16:0/2558416795 learned_addr learned my addr 172.21.15.16:0/2558416795 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061792802 0 0) 0x7f29d4156670 con 0x7f29d4155a20 2024-08-17T02:45:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29b80034e0 con 0x7f29d4155a20 2024-08-17T02:45:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29cbe00640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d406df00 0x7f29d40adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2974757233 0 0) 0x7f29b80034e0 con 0x7f29d4155a20 2024-08-17T02:45:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d406df00 msgr2=0x7f29d40adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29d2000640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d406df00 0x7f29d40adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 >> v1:172.21.15.142:6789/0 conn(0x7f29d4155df0 legacy=0x7f29d4156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d406e8c0 con 0x7f29d4155a20 2024-08-17T02:45:48.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.298+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7f29c4001470 con 0x7f29d4155a20 2024-08-17T02:45:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 >> v1:172.21.15.16:6789/0 conn(0x7f29d4155a20 legacy=0x7f29d406d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d2000640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29b8003a00 0x7f29b8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d2000640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d406df00 0x7f29b8004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d2000640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b8004d10 0x7f29b8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b800bb90 con 0x7f29b8004d10 2024-08-17T02:45:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b800bd00 con 0x7f29b8003a00 2024-08-17T02:45:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b800bec0 con 0x7f29d406df00 2024-08-17T02:45:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d3400640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b8004d10 0x7f29b8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d2a00640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29b8003a00 0x7f29b8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29cbe00640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d406df00 0x7f29b8004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d3400640 1 -- 172.21.15.16:0/2558416795 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d406df00 msgr2=0x7f29b8004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d3400640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d406df00 0x7f29b8004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d3400640 1 -- 172.21.15.16:0/2558416795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29b8003a00 msgr2=0x7f29b8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d3400640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29b8003a00 0x7f29b8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d3400640 1 -- 172.21.15.16:0/2558416795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b80034e0 con 0x7f29b8004d10 2024-08-17T02:45:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.299+0000 7f29d2a00640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29b8003a00 0x7f29b8003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.300+0000 7f29d3400640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b8004d10 0x7f29b8005160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f29d4156230 tx=0x7f29cc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1cf66d5079a5b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.300+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc02ac30 con 0x7f29b8004d10 2024-08-17T02:45:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.300+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc02ad90 con 0x7f29b8004d10 2024-08-17T02:45:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.300+0000 7f29d2000640 1 -- 172.21.15.16:0/2558416795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc032680 con 0x7f29b8004d10 2024-08-17T02:45:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.301+0000 7f29d9c00640 1 -- 172.21.15.16:0/2558416795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b8004d10 msgr2=0x7f29b8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.301+0000 7f29d9c00640 1 --2- 172.21.15.16:0/2558416795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29b8004d10 0x7f29b8005160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f29d4156230 tx=0x7f29cc007f90 comp rx=0 tx=0).stop 2024-08-17T02:45:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.301+0000 7f29d3400640 1 -- 172.21.15.16:0/2558416795 reap_dead start 2024-08-17T02:45:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.301+0000 7f29d9c00640 1 -- 172.21.15.16:0/2558416795 shutdown_connections 2024-08-17T02:45:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.301+0000 7f29d9c00640 1 -- 172.21.15.16:0/2558416795 >> 172.21.15.16:0/2558416795 conn(0x7f29d40cb9b0 msgr2=0x7f29d40cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.301+0000 7f29d9c00640 1 -- 172.21.15.16:0/2558416795 shutdown_connections 2024-08-17T02:45:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.301+0000 7f29d9c00640 1 -- 172.21.15.16:0/2558416795 wait complete. 2024-08-17T02:45:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.303+0000 7f29d9c00640 1 Processor -- start 2024-08-17T02:45:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.303+0000 7f29d9c00640 1 -- start start 2024-08-17T02:45:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.303+0000 7f29d9c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29b8004d10 0x7f29d416a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.303+0000 7f29d9c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d40c3080 0x7f29d4150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.303+0000 7f29d9c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29d4169860 0x7f29d40ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29d9c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40ac560 con 0x7f29d4169860 2024-08-17T02:45:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29d9c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40ab990 con 0x7f29b8004d10 2024-08-17T02:45:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29d9c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40abc30 con 0x7f29d40c3080 2024-08-17T02:45:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29d4169860 0x7f29d40ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29d4169860 0x7f29d40ad7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36468/0 (socket says 172.21.15.16:36468) 2024-08-17T02:45:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29d3400640 1 -- 172.21.15.16:0/3355023912 learned_addr learned my addr 172.21.15.16:0/3355023912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29cbe00640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d40c3080 0x7f29d4150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29d2a00640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29b8004d10 0x7f29d416a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29d3400640 1 -- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d40c3080 msgr2=0x7f29d4150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29d3400640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d40c3080 0x7f29d4150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29d3400640 1 -- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29b8004d10 msgr2=0x7f29d416a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29d3400640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29b8004d10 0x7f29d416a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.304+0000 7f29d3400640 1 -- 172.21.15.16:0/3355023912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d40abf10 con 0x7f29d4169860 2024-08-17T02:45:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.305+0000 7f29d3400640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29d4169860 0x7f29d40ad7f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f29cc02a7f0 tx=0x7f29cc031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.305+0000 7f29ca000640 1 -- 172.21.15.16:0/3355023912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc02ac30 con 0x7f29d4169860 2024-08-17T02:45:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.305+0000 7f29ca000640 1 -- 172.21.15.16:0/3355023912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc032e70 con 0x7f29d4169860 2024-08-17T02:45:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.305+0000 7f29ca000640 1 -- 172.21.15.16:0/3355023912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc031890 con 0x7f29d4169860 2024-08-17T02:45:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.305+0000 7f29d9c00640 1 -- 172.21.15.16:0/3355023912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d40ac1f0 con 0x7f29d4169860 2024-08-17T02:45:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.305+0000 7f29d9c00640 1 -- 172.21.15.16:0/3355023912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d4150bd0 con 0x7f29d4169860 2024-08-17T02:45:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.306+0000 7f29d9c00640 1 -- 172.21.15.16:0/3355023912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d40ac710 con 0x7f29d4169860 2024-08-17T02:45:48.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.307+0000 7f29ca000640 1 -- 172.21.15.16:0/3355023912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f29cc031cc0 con 0x7f29d4169860 2024-08-17T02:45:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.308+0000 7f29ca000640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f29b0065180 0x7f29b0067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.308+0000 7f29ca000640 1 -- 172.21.15.16:0/3355023912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f29cc0a1ea0 con 0x7f29d4169860 2024-08-17T02:45:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.308+0000 7f29d2a00640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f29b0065180 0x7f29b0067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.309+0000 7f29d2a00640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f29b0065180 0x7f29b0067630 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f29c4004d00 tx=0x7f29c4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.312+0000 7f29ca000640 1 -- 172.21.15.16:0/3355023912 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f29d40ac710 con 0x7f29d4169860 2024-08-17T02:45:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.499+0000 7f29d9c00640 1 -- 172.21.15.16:0/3355023912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29d4064db0 con 0x7f29d4169860 2024-08-17T02:45:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.500+0000 7f29ca000640 1 -- 172.21.15.16:0/3355023912 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f29cc06ff70 con 0x7f29d4169860 2024-08-17T02:45:48.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:48.501 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.502+0000 7f29d9c00640 1 -- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f29b0065180 msgr2=0x7f29b0067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.502+0000 7f29d9c00640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f29b0065180 0x7f29b0067630 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f29c4004d00 tx=0x7f29c4009040 comp rx=0 tx=0).stop 2024-08-17T02:45:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.502+0000 7f29d9c00640 1 -- 172.21.15.16:0/3355023912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29d4169860 msgr2=0x7f29d40ad7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.502+0000 7f29d9c00640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29d4169860 0x7f29d40ad7f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f29cc02a7f0 tx=0x7f29cc031000 comp rx=0 tx=0).stop 2024-08-17T02:45:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.502+0000 7f29d9c00640 1 -- 172.21.15.16:0/3355023912 shutdown_connections 2024-08-17T02:45:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.502+0000 7f29d9c00640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f29b0065180 0x7f29b0067630 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.502+0000 7f29d9c00640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29d4169860 0x7f29d40ad7f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.502+0000 7f29d9c00640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f29d40c3080 0x7f29d4150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.502+0000 7f29d9c00640 1 --2- 172.21.15.16:0/3355023912 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29b8004d10 0x7f29d416a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.502+0000 7f29d9c00640 1 -- 172.21.15.16:0/3355023912 >> 172.21.15.16:0/3355023912 conn(0x7f29d40cb9b0 msgr2=0x7f29d41585d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.502+0000 7f29d9c00640 1 -- 172.21.15.16:0/3355023912 shutdown_connections 2024-08-17T02:45:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.502+0000 7f29d9c00640 1 -- 172.21.15.16:0/3355023912 wait complete. 2024-08-17T02:45:48.518 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:45:14.291903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:45:18.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:45:18.430718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:45:48.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:45:48.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.674+0000 7f4783e00640 1 Processor -- start 2024-08-17T02:45:48.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.674+0000 7f4783e00640 1 -- start start 2024-08-17T02:45:48.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.676+0000 7f4783e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f4784154ce0 0x7f47841550b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.676+0000 7f4783e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47840c7940 0x7f47840c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.676+0000 7f4783e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47840afbe0 0x7f47840b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.676+0000 7f4783e00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47840b05c0 con 0x7f47840afbe0 2024-08-17T02:45:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.676+0000 7f4783e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4784159a60 con 0x7f4784154ce0 2024-08-17T02:45:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.676+0000 7f4783e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4784158bb0 con 0x7f47840c7940 2024-08-17T02:45:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.676+0000 7f4783400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47840afbe0 0x7f47840b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.676+0000 7f4783400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47840afbe0 0x7f47840b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36484/0 (socket says 172.21.15.16:36484) 2024-08-17T02:45:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.676+0000 7f4783400640 1 -- 172.21.15.16:0/543786166 learned_addr learned my addr 172.21.15.16:0/543786166 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.676+0000 7f477be00640 1 --2- 172.21.15.16:0/543786166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47840c7940 0x7f47840c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.676+0000 7f4782a00640 1 --2- 172.21.15.16:0/543786166 >> v2:172.21.15.142:3300/0 conn(0x7f4784154ce0 0x7f47841550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.677+0000 7f4783400640 1 -- 172.21.15.16:0/543786166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47840c7940 msgr2=0x7f47840c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.677+0000 7f4783400640 1 --2- 172.21.15.16:0/543786166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47840c7940 0x7f47840c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.677+0000 7f4783400640 1 -- 172.21.15.16:0/543786166 >> v2:172.21.15.142:3300/0 conn(0x7f4784154ce0 msgr2=0x7f47841550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.677+0000 7f4783400640 1 --2- 172.21.15.16:0/543786166 >> v2:172.21.15.142:3300/0 conn(0x7f4784154ce0 0x7f47841550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.677+0000 7f4783400640 1 -- 172.21.15.16:0/543786166 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4784158e50 con 0x7f47840afbe0 2024-08-17T02:45:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.678+0000 7f4783400640 1 --2- 172.21.15.16:0/543786166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47840afbe0 0x7f47840b0080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f478406bba0 tx=0x7f477c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117ca10ad46f81c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.678+0000 7f4782000640 1 -- 172.21.15.16:0/543786166 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c00b2e0 con 0x7f47840afbe0 2024-08-17T02:45:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.678+0000 7f4782000640 1 -- 172.21.15.16:0/543786166 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477c00b440 con 0x7f47840afbe0 2024-08-17T02:45:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.678+0000 7f4782000640 1 -- 172.21.15.16:0/543786166 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c009840 con 0x7f47840afbe0 2024-08-17T02:45:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.678+0000 7f4783e00640 1 -- 172.21.15.16:0/543786166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47840afbe0 msgr2=0x7f47840b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.678+0000 7f4783e00640 1 --2- 172.21.15.16:0/543786166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47840afbe0 0x7f47840b0080 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f478406bba0 tx=0x7f477c00ba50 comp rx=0 tx=0).stop 2024-08-17T02:45:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.678+0000 7f4783e00640 1 -- 172.21.15.16:0/543786166 shutdown_connections 2024-08-17T02:45:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.678+0000 7f4783e00640 1 --2- 172.21.15.16:0/543786166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47840afbe0 0x7f47840b0080 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.678+0000 7f4783e00640 1 --2- 172.21.15.16:0/543786166 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f47840c7940 0x7f47840c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.678+0000 7f4783e00640 1 --2- 172.21.15.16:0/543786166 >> v2:172.21.15.142:3300/0 conn(0x7f4784154ce0 0x7f47841550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.678+0000 7f4783e00640 1 -- 172.21.15.16:0/543786166 >> 172.21.15.16:0/543786166 conn(0x7f4784067750 msgr2=0x7f478406c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.679+0000 7f4783e00640 1 -- 172.21.15.16:0/543786166 shutdown_connections 2024-08-17T02:45:48.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.679+0000 7f4783e00640 1 -- 172.21.15.16:0/543786166 wait complete. 2024-08-17T02:45:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.680+0000 7f4783e00640 1 Processor -- start 2024-08-17T02:45:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.680+0000 7f4783e00640 1 -- start start 2024-08-17T02:45:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.681+0000 7f4783e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47840afbe0 0x7f47840ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.681+0000 7f4783e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47840c7940 0x7f47840c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.681+0000 7f4783e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4784154ce0 0x7f47840ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.681+0000 7f4783e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4784068120 con 0x7f47840c7940 2024-08-17T02:45:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.681+0000 7f4783e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47840ac0f0 con 0x7f47840afbe0 2024-08-17T02:45:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.681+0000 7f4783e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47840c8ae0 con 0x7f4784154ce0 2024-08-17T02:45:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.681+0000 7f477be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47840c7940 0x7f47840c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.681+0000 7f477be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47840c7940 0x7f47840c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36496/0 (socket says 172.21.15.16:36496) 2024-08-17T02:45:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.681+0000 7f477be00640 1 -- 172.21.15.16:0/1305391165 learned_addr learned my addr 172.21.15.16:0/1305391165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.681+0000 7f4783400640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4784154ce0 0x7f47840ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.681+0000 7f4782a00640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47840afbe0 0x7f47840ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.682+0000 7f477be00640 1 -- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4784154ce0 msgr2=0x7f47840ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.682+0000 7f477be00640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4784154ce0 0x7f47840ca130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.682+0000 7f477be00640 1 -- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47840afbe0 msgr2=0x7f47840ac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.682+0000 7f477be00640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47840afbe0 0x7f47840ac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.682+0000 7f477be00640 1 -- 172.21.15.16:0/1305391165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c00bcf0 con 0x7f47840c7940 2024-08-17T02:45:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.682+0000 7f4783400640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4784154ce0 0x7f47840ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.682+0000 7f477be00640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47840c7940 0x7f47840c85a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e8d0 tx=0x7f476c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.682+0000 7f477a000640 1 -- 172.21.15.16:0/1305391165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c00eec0 con 0x7f47840c7940 2024-08-17T02:45:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.682+0000 7f477a000640 1 -- 172.21.15.16:0/1305391165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c00c8e0 con 0x7f47840c7940 2024-08-17T02:45:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.682+0000 7f477a000640 1 -- 172.21.15.16:0/1305391165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c00cc80 con 0x7f47840c7940 2024-08-17T02:45:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.682+0000 7f4783e00640 1 -- 172.21.15.16:0/1305391165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4784169110 con 0x7f47840c7940 2024-08-17T02:45:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.682+0000 7f4783e00640 1 -- 172.21.15.16:0/1305391165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47841696d0 con 0x7f47840c7940 2024-08-17T02:45:48.685 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.684+0000 7f4783e00640 1 -- 172.21.15.16:0/1305391165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4784071050 con 0x7f47840c7940 2024-08-17T02:45:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.684+0000 7f477a000640 1 -- 172.21.15.16:0/1305391165 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f476c013680 con 0x7f47840c7940 2024-08-17T02:45:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.685+0000 7f477a000640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4760065150 0x7f4760067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:48.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.686+0000 7f477a000640 1 -- 172.21.15.16:0/1305391165 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f476c0822d0 con 0x7f47840c7940 2024-08-17T02:45:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.686+0000 7f4782a00640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4760065150 0x7f4760067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:48.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.687+0000 7f4782a00640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4760065150 0x7f4760067600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4774009d20 tx=0x7f47740090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:48.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.689+0000 7f477a000640 1 -- 172.21.15.16:0/1305391165 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f476c0503a0 con 0x7f47840c7940 2024-08-17T02:45:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.875+0000 7f4783e00640 1 -- 172.21.15.16:0/1305391165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47840ad170 con 0x7f47840c7940 2024-08-17T02:45:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.875+0000 7f477a000640 1 -- 172.21.15.16:0/1305391165 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f476c0539c0 con 0x7f47840c7940 2024-08-17T02:45:48.877 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-17T02:45:48.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.878+0000 7f4783e00640 1 -- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4760065150 msgr2=0x7f4760067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.878+0000 7f4783e00640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4760065150 0x7f4760067600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4774009d20 tx=0x7f47740090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.878+0000 7f4783e00640 1 -- 172.21.15.16:0/1305391165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47840c7940 msgr2=0x7f47840c85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:48.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.878+0000 7f4783e00640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47840c7940 0x7f47840c85a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e8d0 tx=0x7f476c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.878+0000 7f4783e00640 1 -- 172.21.15.16:0/1305391165 shutdown_connections 2024-08-17T02:45:48.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.878+0000 7f4783e00640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4760065150 0x7f4760067600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.878+0000 7f4783e00640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4784154ce0 0x7f47840ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.878+0000 7f4783e00640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47840c7940 0x7f47840c85a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.878+0000 7f4783e00640 1 --2- 172.21.15.16:0/1305391165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47840afbe0 0x7f47840ac720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.878+0000 7f4783e00640 1 -- 172.21.15.16:0/1305391165 >> 172.21.15.16:0/1305391165 conn(0x7f4784067750 msgr2=0x7f478414fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.878+0000 7f4783e00640 1 -- 172.21.15.16:0/1305391165 shutdown_connections 2024-08-17T02:45:48.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:48.878+0000 7f4783e00640 1 -- 172.21.15.16:0/1305391165 wait complete. 2024-08-17T02:45:48.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:45:49.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.047+0000 7fe726400640 1 Processor -- start 2024-08-17T02:45:49.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.047+0000 7fe726400640 1 -- start start 2024-08-17T02:45:49.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.048+0000 7fe726400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe7200af540 0x7fe7200af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.048+0000 7fe726400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe72015c980 0x7fe72006b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.048+0000 7fe726400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7200b1410 0x7fe720158820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.048+0000 7fe726400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72006ac60 con 0x7fe7200b1410 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.048+0000 7fe726400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7200b1860 con 0x7fe7200af540 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.048+0000 7fe726400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe720069d70 con 0x7fe72015c980 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.049+0000 7fe71fe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7200b1410 0x7fe720158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.049+0000 7fe71fe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7200b1410 0x7fe720158820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36502/0 (socket says 172.21.15.16:36502) 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.049+0000 7fe71fe00640 1 -- 172.21.15.16:0/1279763231 learned_addr learned my addr 172.21.15.16:0/1279763231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.049+0000 7fe71f400640 1 --2- 172.21.15.16:0/1279763231 >> v2:172.21.15.142:3300/0 conn(0x7fe7200af540 0x7fe7200af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.049+0000 7fe717e00640 1 --2- 172.21.15.16:0/1279763231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe72015c980 0x7fe72006b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.049+0000 7fe71fe00640 1 -- 172.21.15.16:0/1279763231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe72015c980 msgr2=0x7fe72006b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.049+0000 7fe71fe00640 1 --2- 172.21.15.16:0/1279763231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe72015c980 0x7fe72006b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.049+0000 7fe71fe00640 1 -- 172.21.15.16:0/1279763231 >> v2:172.21.15.142:3300/0 conn(0x7fe7200af540 msgr2=0x7fe7200af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:49.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.049+0000 7fe71fe00640 1 --2- 172.21.15.16:0/1279763231 >> v2:172.21.15.142:3300/0 conn(0x7fe7200af540 0x7fe7200af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.049+0000 7fe71fe00640 1 -- 172.21.15.16:0/1279763231 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72006a050 con 0x7fe7200b1410 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.050+0000 7fe71fe00640 1 --2- 172.21.15.16:0/1279763231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7200b1410 0x7fe720158820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe7200644b0 tx=0x7fe718009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800099a9ced07656 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.051+0000 7fe71ea00640 1 -- 172.21.15.16:0/1279763231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718031400 con 0x7fe7200b1410 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.051+0000 7fe71ea00640 1 -- 172.21.15.16:0/1279763231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71802d050 con 0x7fe7200b1410 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.051+0000 7fe71ea00640 1 -- 172.21.15.16:0/1279763231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7180317d0 con 0x7fe7200b1410 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.051+0000 7fe726400640 1 -- 172.21.15.16:0/1279763231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7200b1410 msgr2=0x7fe720158820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.051+0000 7fe726400640 1 --2- 172.21.15.16:0/1279763231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7200b1410 0x7fe720158820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe7200644b0 tx=0x7fe718009840 comp rx=0 tx=0).stop 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.051+0000 7fe726400640 1 -- 172.21.15.16:0/1279763231 shutdown_connections 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.051+0000 7fe726400640 1 --2- 172.21.15.16:0/1279763231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7200b1410 0x7fe720158820 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.051+0000 7fe726400640 1 --2- 172.21.15.16:0/1279763231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe72015c980 0x7fe72006b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.051+0000 7fe726400640 1 --2- 172.21.15.16:0/1279763231 >> v2:172.21.15.142:3300/0 conn(0x7fe7200af540 0x7fe7200af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.051+0000 7fe726400640 1 -- 172.21.15.16:0/1279763231 >> 172.21.15.16:0/1279763231 conn(0x7fe7200cbdc0 msgr2=0x7fe7200cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.051+0000 7fe726400640 1 -- 172.21.15.16:0/1279763231 shutdown_connections 2024-08-17T02:45:49.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.051+0000 7fe726400640 1 -- 172.21.15.16:0/1279763231 wait complete. 2024-08-17T02:45:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.053+0000 7fe726400640 1 Processor -- start 2024-08-17T02:45:49.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.053+0000 7fe726400640 1 -- start start 2024-08-17T02:45:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe726400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7200af540 0x7fe720148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe726400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7200b1410 0x7fe7200c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe726400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe72015c980 0x7fe72015b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe726400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720149360 con 0x7fe7200b1410 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe726400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720158ac0 con 0x7fe7200af540 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe726400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe720158770 con 0x7fe72015c980 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe717e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7200b1410 0x7fe7200c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe717e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7200b1410 0x7fe7200c0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36504/0 (socket says 172.21.15.16:36504) 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe717e00640 1 -- 172.21.15.16:0/3791464111 learned_addr learned my addr 172.21.15.16:0/3791464111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe71fe00640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe72015c980 0x7fe72015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe71f400640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7200af540 0x7fe720148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe717e00640 1 -- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe72015c980 msgr2=0x7fe72015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe717e00640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe72015c980 0x7fe72015b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe717e00640 1 -- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7200af540 msgr2=0x7fe720148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe717e00640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7200af540 0x7fe720148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.054+0000 7fe717e00640 1 -- 172.21.15.16:0/3791464111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe718009ae0 con 0x7fe7200b1410 2024-08-17T02:45:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.055+0000 7fe717e00640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7200b1410 0x7fe7200c0db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe70800d890 tx=0x7fe708008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:49.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.055+0000 7fe716a00640 1 -- 172.21.15.16:0/3791464111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70800de20 con 0x7fe7200b1410 2024-08-17T02:45:49.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.055+0000 7fe716a00640 1 -- 172.21.15.16:0/3791464111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7080098b0 con 0x7fe7200b1410 2024-08-17T02:45:49.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.055+0000 7fe726400640 1 -- 172.21.15.16:0/3791464111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7200c1790 con 0x7fe7200b1410 2024-08-17T02:45:49.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.055+0000 7fe716a00640 1 -- 172.21.15.16:0/3791464111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708009c50 con 0x7fe7200b1410 2024-08-17T02:45:49.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.055+0000 7fe726400640 1 -- 172.21.15.16:0/3791464111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe720146340 con 0x7fe7200b1410 2024-08-17T02:45:49.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.056+0000 7fe726400640 1 -- 172.21.15.16:0/3791464111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7201585d0 con 0x7fe7200b1410 2024-08-17T02:45:49.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.057+0000 7fe716a00640 1 -- 172.21.15.16:0/3791464111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fe708014620 con 0x7fe7200b1410 2024-08-17T02:45:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.058+0000 7fe716a00640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe6fc065150 0x7fe6fc067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.058+0000 7fe716a00640 1 -- 172.21.15.16:0/3791464111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe708082bf0 con 0x7fe7200b1410 2024-08-17T02:45:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.058+0000 7fe71f400640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe6fc065150 0x7fe6fc067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.059+0000 7fe71f400640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe6fc065150 0x7fe6fc067600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe710009c80 tx=0x7fe7100090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.062+0000 7fe716a00640 1 -- 172.21.15.16:0/3791464111 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe7201585d0 con 0x7fe7200b1410 2024-08-17T02:45:49.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.242+0000 7fe726400640 1 -- 172.21.15.16:0/3791464111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe7201585d0 con 0x7fe7200b1410 2024-08-17T02:45:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.242+0000 7fe716a00640 1 -- 172.21.15.16:0/3791464111 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe7201585d0 con 0x7fe7200b1410 2024-08-17T02:45:49.244 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:45:49.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.244+0000 7fe726400640 1 -- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe6fc065150 msgr2=0x7fe6fc067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.244+0000 7fe726400640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe6fc065150 0x7fe6fc067600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe710009c80 tx=0x7fe7100090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.244+0000 7fe726400640 1 -- 172.21.15.16:0/3791464111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7200b1410 msgr2=0x7fe7200c0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.244+0000 7fe726400640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7200b1410 0x7fe7200c0db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe70800d890 tx=0x7fe708008040 comp rx=0 tx=0).stop 2024-08-17T02:45:49.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.244+0000 7fe726400640 1 -- 172.21.15.16:0/3791464111 shutdown_connections 2024-08-17T02:45:49.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.244+0000 7fe726400640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe6fc065150 0x7fe6fc067600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.244+0000 7fe726400640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe72015c980 0x7fe72015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.244+0000 7fe726400640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7200b1410 0x7fe7200c0db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.244+0000 7fe726400640 1 --2- 172.21.15.16:0/3791464111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7200af540 0x7fe720148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.244+0000 7fe726400640 1 -- 172.21.15.16:0/3791464111 >> 172.21.15.16:0/3791464111 conn(0x7fe7200cbdc0 msgr2=0x7fe7200cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:49.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.244+0000 7fe726400640 1 -- 172.21.15.16:0/3791464111 shutdown_connections 2024-08-17T02:45:49.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.244+0000 7fe726400640 1 -- 172.21.15.16:0/3791464111 wait complete. 2024-08-17T02:45:49.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:45:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.415+0000 7f1952800640 1 Processor -- start 2024-08-17T02:45:49.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.415+0000 7f1952800640 1 -- start start 2024-08-17T02:45:49.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.416+0000 7f1952800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 0x7f194c163b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.416+0000 7f1952800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c0afbb0 con 0x7f194c0d0890 2024-08-17T02:45:49.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.416+0000 7f1952800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c0afe90 con 0x7f194c15c2f0 2024-08-17T02:45:49.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.416+0000 7f1952800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0b1e10 con 0x7f194c0b0f20 2024-08-17T02:45:49.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.416+0000 7f194aa00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f194c0d0890 0x7f194c0b0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52820/0 (socket says 172.21.15.16:52820) 2024-08-17T02:45:49.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.416+0000 7f194aa00640 1 -- 172.21.15.16:0/3785668689 learned_addr learned my addr 172.21.15.16:0/3785668689 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:49.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.417+0000 7f194be00640 1 --2- 172.21.15.16:0/3785668689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 0x7f194c163b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.417+0000 7f194a000640 1 -- 172.21.15.16:0/3785668689 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509310814 0 0) 0x7f194c0afbb0 con 0x7f194c0d0890 2024-08-17T02:45:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.417+0000 7f194a000640 1 -- 172.21.15.16:0/3785668689 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19300034e0 con 0x7f194c0d0890 2024-08-17T02:45:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.417+0000 7f194a000640 1 -- 172.21.15.16:0/3785668689 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478766893 0 0) 0x7f194c0afe90 con 0x7f194c15c2f0 2024-08-17T02:45:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.417+0000 7f194a000640 1 -- 172.21.15.16:0/3785668689 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f194c0afbb0 con 0x7f194c15c2f0 2024-08-17T02:45:49.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.417+0000 7f194be00640 1 -- 172.21.15.16:0/3785668689 >> v1:172.21.15.142:6789/0 conn(0x7f194c15c2f0 legacy=0x7f194c0af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.417+0000 7f194be00640 1 -- 172.21.15.16:0/3785668689 >> v1:172.21.15.16:6789/0 conn(0x7f194c0d0890 legacy=0x7f194c0b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.417+0000 7f194a000640 1 -- 172.21.15.16:0/3785668689 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 348180741 0 0) 0x7f19300034e0 con 0x7f194c0d0890 2024-08-17T02:45:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.417+0000 7f194be00640 1 -- 172.21.15.16:0/3785668689 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0b0170 con 0x7f194c0b0f20 2024-08-17T02:45:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.418+0000 7f194be00640 1 --2- 172.21.15.16:0/3785668689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 0x7f194c163b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1944009e90 tx=0x7f1944009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38701c99af3f9cc server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.419+0000 7f194a000640 1 -- 172.21.15.16:0/3785668689 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1938001470 con 0x7f194c0b0f20 2024-08-17T02:45:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.419+0000 7f194a000640 1 -- 172.21.15.16:0/3785668689 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194402d050 con 0x7f194c0b0f20 2024-08-17T02:45:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.419+0000 7f194a000640 1 -- 172.21.15.16:0/3785668689 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944031630 con 0x7f194c0b0f20 2024-08-17T02:45:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.419+0000 7f1952800640 1 -- 172.21.15.16:0/3785668689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 msgr2=0x7f194c163b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.419+0000 7f1952800640 1 --2- 172.21.15.16:0/3785668689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 0x7f194c163b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1944009e90 tx=0x7f1944009780 comp rx=0 tx=0).stop 2024-08-17T02:45:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.419+0000 7f1952800640 1 -- 172.21.15.16:0/3785668689 shutdown_connections 2024-08-17T02:45:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.419+0000 7f1952800640 1 --2- 172.21.15.16:0/3785668689 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 0x7f194c163b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.419+0000 7f1952800640 1 -- 172.21.15.16:0/3785668689 >> 172.21.15.16:0/3785668689 conn(0x7f194c0cfdb0 msgr2=0x7f194c0d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.419+0000 7f1952800640 1 -- 172.21.15.16:0/3785668689 shutdown_connections 2024-08-17T02:45:49.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.419+0000 7f1952800640 1 -- 172.21.15.16:0/3785668689 wait complete. 2024-08-17T02:45:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.421+0000 7f1952800640 1 Processor -- start 2024-08-17T02:45:49.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.421+0000 7f1952800640 1 -- start start 2024-08-17T02:45:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.422+0000 7f1952800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 0x7f194c15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.422+0000 7f1952800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f194c0d0890 0x7f194c158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.422+0000 7f1952800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c15c2f0 0x7f194c158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.422+0000 7f1952800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0cd140 con 0x7f194c15c2f0 2024-08-17T02:45:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.422+0000 7f1952800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0cc530 con 0x7f194c0d0890 2024-08-17T02:45:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.422+0000 7f1952800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0cc810 con 0x7f194c0b0f20 2024-08-17T02:45:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.422+0000 7f194b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 0x7f194c15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.422+0000 7f194be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c15c2f0 0x7f194c158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.422+0000 7f194b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 0x7f194c15b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:49952/0 (socket says 172.21.15.16:49952) 2024-08-17T02:45:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.422+0000 7f194b400640 1 -- 172.21.15.16:0/2451168358 learned_addr learned my addr 172.21.15.16:0/2451168358 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.422+0000 7f194aa00640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f194c0d0890 0x7f194c158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.423+0000 7f194b400640 1 -- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f194c0d0890 msgr2=0x7f194c158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.423+0000 7f194b400640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f194c0d0890 0x7f194c158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.423+0000 7f194b400640 1 -- 172.21.15.16:0/2451168358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c15c2f0 msgr2=0x7f194c158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.423+0000 7f194b400640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c15c2f0 0x7f194c158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.423+0000 7f194b400640 1 -- 172.21.15.16:0/2451168358 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1944009a20 con 0x7f194c0b0f20 2024-08-17T02:45:49.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.423+0000 7f194b400640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 0x7f194c15b430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f193c00c4e0 tx=0x7f193c007d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:49.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.423+0000 7f1942a00640 1 -- 172.21.15.16:0/2451168358 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c016070 con 0x7f194c0b0f20 2024-08-17T02:45:49.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.423+0000 7f1942a00640 1 -- 172.21.15.16:0/2451168358 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193c011590 con 0x7f194c0b0f20 2024-08-17T02:45:49.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.423+0000 7f1952800640 1 -- 172.21.15.16:0/2451168358 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c0ccaf0 con 0x7f194c0b0f20 2024-08-17T02:45:49.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.423+0000 7f1942a00640 1 -- 172.21.15.16:0/2451168358 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c011990 con 0x7f194c0b0f20 2024-08-17T02:45:49.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.423+0000 7f1952800640 1 -- 172.21.15.16:0/2451168358 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c0cc170 con 0x7f194c0b0f20 2024-08-17T02:45:49.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.424+0000 7f1952800640 1 -- 172.21.15.16:0/2451168358 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c071050 con 0x7f194c0b0f20 2024-08-17T02:45:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.425+0000 7f1942a00640 1 -- 172.21.15.16:0/2451168358 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f193c012420 con 0x7f194c0b0f20 2024-08-17T02:45:49.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.426+0000 7f1942a00640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f192c065150 0x7f192c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.426+0000 7f1942a00640 1 -- 172.21.15.16:0/2451168358 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f193c081e70 con 0x7f194c0b0f20 2024-08-17T02:45:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.427+0000 7f194aa00640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f192c065150 0x7f192c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.427+0000 7f194aa00640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f192c065150 0x7f192c067600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f194c16c260 tx=0x7f1938001b00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:49.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.431+0000 7f1942a00640 1 -- 172.21.15.16:0/2451168358 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f193c04ff40 con 0x7f194c0b0f20 2024-08-17T02:45:49.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.609+0000 7f1952800640 1 -- 172.21.15.16:0/2451168358 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f194c15d5c0 con 0x7f194c0b0f20 2024-08-17T02:45:49.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.610+0000 7f1942a00640 1 -- 172.21.15.16:0/2451168358 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f193c053560 con 0x7f194c0b0f20 2024-08-17T02:45:49.612 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-08-17T02:45:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.612+0000 7f1952800640 1 -- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f192c065150 msgr2=0x7f192c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.612+0000 7f1952800640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f192c065150 0x7f192c067600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f194c16c260 tx=0x7f1938001b00 comp rx=0 tx=0).stop 2024-08-17T02:45:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.612+0000 7f1952800640 1 -- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 msgr2=0x7f194c15b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.612+0000 7f1952800640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 0x7f194c15b430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f193c00c4e0 tx=0x7f193c007d80 comp rx=0 tx=0).stop 2024-08-17T02:45:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.612+0000 7f1952800640 1 -- 172.21.15.16:0/2451168358 shutdown_connections 2024-08-17T02:45:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.612+0000 7f1952800640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f192c065150 0x7f192c067600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.612+0000 7f1952800640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c15c2f0 0x7f194c158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.612+0000 7f1952800640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f194c0d0890 0x7f194c158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.612+0000 7f1952800640 1 --2- 172.21.15.16:0/2451168358 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f194c0b0f20 0x7f194c15b430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.612+0000 7f1952800640 1 -- 172.21.15.16:0/2451168358 >> 172.21.15.16:0/2451168358 conn(0x7f194c0cfdb0 msgr2=0x7f194c15c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.612+0000 7f1952800640 1 -- 172.21.15.16:0/2451168358 shutdown_connections 2024-08-17T02:45:49.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.612+0000 7f1952800640 1 -- 172.21.15.16:0/2451168358 wait complete. 2024-08-17T02:45:49.629 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:49.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:45:49.750 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:45:49.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.780+0000 7f42e0e00640 1 Processor -- start 2024-08-17T02:45:49.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.780+0000 7f42e0e00640 1 -- start start 2024-08-17T02:45:49.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.781+0000 7f42e0e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f42dc158c30 0x7f42dc159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.781+0000 7f42e0e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42dc0cdd90 0x7f42dc0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.781+0000 7f42e0e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42dc0cca90 0x7f42dc0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.781+0000 7f42e0e00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42dc0ccf10 con 0x7f42dc0cdd90 2024-08-17T02:45:49.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.781+0000 7f42e0e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42dc0cea50 con 0x7f42dc158c30 2024-08-17T02:45:49.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.781+0000 7f42e0e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0cab60 con 0x7f42dc0cca90 2024-08-17T02:45:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.781+0000 7f42daa00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42dc0cdd90 0x7f42dc0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.782+0000 7f42daa00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42dc0cdd90 0x7f42dc0cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36514/0 (socket says 172.21.15.16:36514) 2024-08-17T02:45:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.782+0000 7f42daa00640 1 -- 172.21.15.16:0/2374638617 learned_addr learned my addr 172.21.15.16:0/2374638617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.782+0000 7f42db400640 1 --2- 172.21.15.16:0/2374638617 >> v2:172.21.15.142:3300/0 conn(0x7f42dc158c30 0x7f42dc159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.782+0000 7f42dbe00640 1 --2- 172.21.15.16:0/2374638617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42dc0cca90 0x7f42dc0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.782+0000 7f42daa00640 1 -- 172.21.15.16:0/2374638617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42dc0cca90 msgr2=0x7f42dc0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.782+0000 7f42daa00640 1 --2- 172.21.15.16:0/2374638617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42dc0cca90 0x7f42dc0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.782+0000 7f42daa00640 1 -- 172.21.15.16:0/2374638617 >> v2:172.21.15.142:3300/0 conn(0x7f42dc158c30 msgr2=0x7f42dc159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.782+0000 7f42daa00640 1 --2- 172.21.15.16:0/2374638617 >> v2:172.21.15.142:3300/0 conn(0x7f42dc158c30 0x7f42dc159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.782+0000 7f42daa00640 1 -- 172.21.15.16:0/2374638617 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc159000 con 0x7f42dc0cdd90 2024-08-17T02:45:49.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.783+0000 7f42daa00640 1 --2- 172.21.15.16:0/2374638617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc0cdd90 0x7f42dc0cc550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f42dc088b20 tx=0x7f42c80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f594e4c8ae20cb25 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:49.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.783+0000 7f42da000640 1 -- 172.21.15.16:0/2374638617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c802add0 con 0x7f42dc0cdd90 2024-08-17T02:45:49.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.783+0000 7f42da000640 1 -- 172.21.15.16:0/2374638617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c8032440 con 0x7f42dc0cdd90 2024-08-17T02:45:49.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.783+0000 7f42da000640 1 -- 172.21.15.16:0/2374638617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8032820 con 0x7f42dc0cdd90 2024-08-17T02:45:49.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.784+0000 7f42e0e00640 1 -- 172.21.15.16:0/2374638617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc0cdd90 msgr2=0x7f42dc0cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.784+0000 7f42e0e00640 1 --2- 172.21.15.16:0/2374638617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc0cdd90 0x7f42dc0cc550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f42dc088b20 tx=0x7f42c80091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.784+0000 7f42e0e00640 1 -- 172.21.15.16:0/2374638617 shutdown_connections 2024-08-17T02:45:49.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.784+0000 7f42e0e00640 1 --2- 172.21.15.16:0/2374638617 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42dc0cca90 0x7f42dc0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.784+0000 7f42e0e00640 1 --2- 172.21.15.16:0/2374638617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc0cdd90 0x7f42dc0cc550 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.784+0000 7f42e0e00640 1 --2- 172.21.15.16:0/2374638617 >> v2:172.21.15.142:3300/0 conn(0x7f42dc158c30 0x7f42dc159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.784+0000 7f42e0e00640 1 -- 172.21.15.16:0/2374638617 >> 172.21.15.16:0/2374638617 conn(0x7f42dc098e20 msgr2=0x7f42dc06eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:49.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.784+0000 7f42e0e00640 1 -- 172.21.15.16:0/2374638617 shutdown_connections 2024-08-17T02:45:49.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.784+0000 7f42e0e00640 1 -- 172.21.15.16:0/2374638617 wait complete. 2024-08-17T02:45:49.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.786+0000 7f42e0e00640 1 Processor -- start 2024-08-17T02:45:49.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.786+0000 7f42e0e00640 1 -- start start 2024-08-17T02:45:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.787+0000 7f42e0e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42dc0cca90 0x7f42dc149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.787+0000 7f42e0e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc0cdd90 0x7f42dc149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.787+0000 7f42e0e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc158c30 0x7f42dc09a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.787+0000 7f42e0e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc14b140 con 0x7f42dc158c30 2024-08-17T02:45:49.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.787+0000 7f42e0e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc099a20 con 0x7f42dc0cdd90 2024-08-17T02:45:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.787+0000 7f42e0e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc14a530 con 0x7f42dc0cca90 2024-08-17T02:45:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.787+0000 7f42dbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc158c30 0x7f42dc09a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.787+0000 7f42dbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc158c30 0x7f42dc09a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36526/0 (socket says 172.21.15.16:36526) 2024-08-17T02:45:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.787+0000 7f42dbe00640 1 -- 172.21.15.16:0/4211158054 learned_addr learned my addr 172.21.15.16:0/4211158054 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.787+0000 7f42db400640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42dc0cca90 0x7f42dc149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.787+0000 7f42daa00640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc0cdd90 0x7f42dc149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.788+0000 7f42dbe00640 1 -- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42dc0cca90 msgr2=0x7f42dc149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.788+0000 7f42dbe00640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42dc0cca90 0x7f42dc149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.788+0000 7f42dbe00640 1 -- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc0cdd90 msgr2=0x7f42dc149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.788+0000 7f42dbe00640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc0cdd90 0x7f42dc149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.788+0000 7f42dbe00640 1 -- 172.21.15.16:0/4211158054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c8009450 con 0x7f42dc158c30 2024-08-17T02:45:49.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.788+0000 7f42dbe00640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc158c30 0x7f42dc09a0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009fa0 tx=0x7f42d4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:49.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.788+0000 7f42d2a00640 1 -- 172.21.15.16:0/4211158054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d400ee60 con 0x7f42dc158c30 2024-08-17T02:45:49.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.788+0000 7f42d2a00640 1 -- 172.21.15.16:0/4211158054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d4014420 con 0x7f42dc158c30 2024-08-17T02:45:49.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.788+0000 7f42d2a00640 1 -- 172.21.15.16:0/4211158054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d40147e0 con 0x7f42dc158c30 2024-08-17T02:45:49.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.788+0000 7f42e0e00640 1 -- 172.21.15.16:0/4211158054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc14a810 con 0x7f42dc158c30 2024-08-17T02:45:49.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.788+0000 7f42e0e00640 1 -- 172.21.15.16:0/4211158054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc14add0 con 0x7f42dc158c30 2024-08-17T02:45:49.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.790+0000 7f42e0e00640 1 -- 172.21.15.16:0/4211158054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc14b2b0 con 0x7f42dc158c30 2024-08-17T02:45:49.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.790+0000 7f42d2a00640 1 -- 172.21.15.16:0/4211158054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f42d4014b50 con 0x7f42dc158c30 2024-08-17T02:45:49.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.791+0000 7f42d2a00640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f42bc065150 0x7f42bc067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:49.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.791+0000 7f42d2a00640 1 -- 172.21.15.16:0/4211158054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f42d4082b40 con 0x7f42dc158c30 2024-08-17T02:45:49.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.791+0000 7f42db400640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f42bc065150 0x7f42bc067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:49.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.792+0000 7f42db400640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f42bc065150 0x7f42bc067600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00bd40 tx=0x7f42cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:49.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.795+0000 7f42d2a00640 1 -- 172.21.15.16:0/4211158054 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f42d4050cb0 con 0x7f42dc158c30 2024-08-17T02:45:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.979+0000 7f42e0e00640 1 -- 172.21.15.16:0/4211158054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42dc071050 con 0x7f42dc158c30 2024-08-17T02:45:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.980+0000 7f42d2a00640 1 -- 172.21.15.16:0/4211158054 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f42d40542d0 con 0x7f42dc158c30 2024-08-17T02:45:49.981 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:49.981 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.982+0000 7f42e0e00640 1 -- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f42bc065150 msgr2=0x7f42bc067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.982+0000 7f42e0e00640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f42bc065150 0x7f42bc067600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f42cc00bd40 tx=0x7f42cc008040 comp rx=0 tx=0).stop 2024-08-17T02:45:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.982+0000 7f42e0e00640 1 -- 172.21.15.16:0/4211158054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc158c30 msgr2=0x7f42dc09a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.982+0000 7f42e0e00640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc158c30 0x7f42dc09a0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009fa0 tx=0x7f42d4009900 comp rx=0 tx=0).stop 2024-08-17T02:45:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.982+0000 7f42e0e00640 1 -- 172.21.15.16:0/4211158054 shutdown_connections 2024-08-17T02:45:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.982+0000 7f42e0e00640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f42bc065150 0x7f42bc067600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.982+0000 7f42e0e00640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42dc158c30 0x7f42dc09a0c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.982+0000 7f42e0e00640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f42dc0cdd90 0x7f42dc149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.982+0000 7f42e0e00640 1 --2- 172.21.15.16:0/4211158054 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f42dc0cca90 0x7f42dc149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.982+0000 7f42e0e00640 1 -- 172.21.15.16:0/4211158054 >> 172.21.15.16:0/4211158054 conn(0x7f42dc098e20 msgr2=0x7f42dc068b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.982+0000 7f42e0e00640 1 -- 172.21.15.16:0/4211158054 shutdown_connections 2024-08-17T02:45:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:49.982+0000 7f42e0e00640 1 -- 172.21.15.16:0/4211158054 wait complete. 2024-08-17T02:45:49.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:45:14.291903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:45:18.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:45:18.430718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:45:50.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:45:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.153+0000 7f3a5ee00640 1 Processor -- start 2024-08-17T02:45:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.153+0000 7f3a5ee00640 1 -- start start 2024-08-17T02:45:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.154+0000 7f3a5ee00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3a580accf0 0x7f3a580b1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.154+0000 7f3a5ee00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a580ae540 con 0x7f3a580afd90 2024-08-17T02:45:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.154+0000 7f3a5ee00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a580ad930 con 0x7f3a58159610 2024-08-17T02:45:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.154+0000 7f3a5ee00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a580adc10 con 0x7f3a580accf0 2024-08-17T02:45:50.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.154+0000 7f3a5ce00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3a580afd90 0x7f3a58157690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52832/0 (socket says 172.21.15.16:52832) 2024-08-17T02:45:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.154+0000 7f3a5ce00640 1 -- 172.21.15.16:0/4228319497 learned_addr learned my addr 172.21.15.16:0/4228319497 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57e00640 1 --2- 172.21.15.16:0/4228319497 >> v2:172.21.15.142:3300/0 conn(0x7f3a580accf0 0x7f3a580b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360869246 0 0) 0x7f3a580ae540 con 0x7f3a580afd90 2024-08-17T02:45:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a3c0034e0 con 0x7f3a580afd90 2024-08-17T02:45:50.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3578675987 0 0) 0x7f3a3c0034e0 con 0x7f3a580afd90 2024-08-17T02:45:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 >> v2:172.21.15.142:3300/0 conn(0x7f3a580accf0 msgr2=0x7f3a580b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 --2- 172.21.15.16:0/4228319497 >> v2:172.21.15.142:3300/0 conn(0x7f3a580accf0 0x7f3a580b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 >> v1:172.21.15.142:6789/0 conn(0x7f3a58159610 legacy=0x7f3a580af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a580adef0 con 0x7f3a580afd90 2024-08-17T02:45:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7f3a500017f0 con 0x7f3a580afd90 2024-08-17T02:45:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57e00640 1 --2- 172.21.15.16:0/4228319497 >> v2:172.21.15.142:3300/0 conn(0x7f3a580accf0 0x7f3a580b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 >> v1:172.21.15.16:6789/0 conn(0x7f3a580afd90 legacy=0x7f3a58157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 --2- 172.21.15.16:0/4228319497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3c003a20 0x7f3a3c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 --2- 172.21.15.16:0/4228319497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a3c004780 0x7f3a3c004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 --2- 172.21.15.16:0/4228319497 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a3c005180 0x7f3a3c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580adc10 con 0x7f3a3c003a20 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c00c0a0 con 0x7f3a3c004780 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.155+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c00c260 con 0x7f3a3c005180 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.156+0000 7f3a57e00640 1 --2- 172.21.15.16:0/4228319497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3c003a20 0x7f3a3c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.156+0000 7f3a5ce00640 1 --2- 172.21.15.16:0/4228319497 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a3c005180 0x7f3a3c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.156+0000 7f3a4fe00640 1 --2- 172.21.15.16:0/4228319497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a3c004780 0x7f3a3c004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.156+0000 7f3a57e00640 1 -- 172.21.15.16:0/4228319497 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a3c005180 msgr2=0x7f3a3c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.156+0000 7f3a57e00640 1 --2- 172.21.15.16:0/4228319497 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a3c005180 0x7f3a3c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.156+0000 7f3a57e00640 1 -- 172.21.15.16:0/4228319497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a3c004780 msgr2=0x7f3a3c004bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.156+0000 7f3a57e00640 1 --2- 172.21.15.16:0/4228319497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a3c004780 0x7f3a3c004bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.156+0000 7f3a57e00640 1 -- 172.21.15.16:0/4228319497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c0034e0 con 0x7f3a3c003a20 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.156+0000 7f3a4fe00640 1 --2- 172.21.15.16:0/4228319497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a3c004780 0x7f3a3c004bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.156+0000 7f3a5ce00640 1 -- 172.21.15.16:0/4228319497 reap_dead start 2024-08-17T02:45:50.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.157+0000 7f3a57e00640 1 --2- 172.21.15.16:0/4228319497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3c003a20 0x7f3a3c003e30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009d60 tx=0x7f3a480090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b53fe7481cb0c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:50.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.157+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48031400 con 0x7f3a3c003a20 2024-08-17T02:45:50.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.157+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4802d050 con 0x7f3a3c003a20 2024-08-17T02:45:50.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.157+0000 7f3a57400640 1 -- 172.21.15.16:0/4228319497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48031770 con 0x7f3a3c003a20 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.158+0000 7f3a5ee00640 1 -- 172.21.15.16:0/4228319497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3c003a20 msgr2=0x7f3a3c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.158+0000 7f3a5ee00640 1 --2- 172.21.15.16:0/4228319497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3c003a20 0x7f3a3c003e30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009d60 tx=0x7f3a480090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.158+0000 7f3a5ee00640 1 -- 172.21.15.16:0/4228319497 shutdown_connections 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.158+0000 7f3a5ee00640 1 --2- 172.21.15.16:0/4228319497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a3c003a20 0x7f3a3c003e30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.158+0000 7f3a5ee00640 1 -- 172.21.15.16:0/4228319497 >> 172.21.15.16:0/4228319497 conn(0x7f3a58067750 msgr2=0x7f3a580cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.158+0000 7f3a5ee00640 1 -- 172.21.15.16:0/4228319497 shutdown_connections 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.158+0000 7f3a5ee00640 1 -- 172.21.15.16:0/4228319497 wait complete. 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.159+0000 7f3a5ee00640 1 Processor -- start 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.160+0000 7f3a5ee00640 1 -- start start 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.160+0000 7f3a5ee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a3c003a20 0x7f3a58151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.160+0000 7f3a5ee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a58156ac0 0x7f3a580a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.160+0000 7f3a5ee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a58168db0 0x7f3a5814f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a5ee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a58148fe0 con 0x7f3a58156ac0 2024-08-17T02:45:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a5ee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5814e0f0 con 0x7f3a3c003a20 2024-08-17T02:45:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a5ee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a58169650 con 0x7f3a58168db0 2024-08-17T02:45:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a4fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a58156ac0 0x7f3a580a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a4fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a58156ac0 0x7f3a580a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36546/0 (socket says 172.21.15.16:36546) 2024-08-17T02:45:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a4fe00640 1 -- 172.21.15.16:0/3403737063 learned_addr learned my addr 172.21.15.16:0/3403737063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a5ce00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a58168db0 0x7f3a5814f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a57e00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a3c003a20 0x7f3a58151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a4fe00640 1 -- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a58168db0 msgr2=0x7f3a5814f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a4fe00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a58168db0 0x7f3a5814f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a4fe00640 1 -- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a3c003a20 msgr2=0x7f3a58151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a4fe00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a3c003a20 0x7f3a58151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a4fe00640 1 -- 172.21.15.16:0/3403737063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a58169930 con 0x7f3a58156ac0 2024-08-17T02:45:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a5ce00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a58168db0 0x7f3a5814f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.161+0000 7f3a4fe00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a58156ac0 0x7f3a580a98e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3a40001eb0 tx=0x7f3a40009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.162+0000 7f3a4f400640 1 -- 172.21.15.16:0/3403737063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40014480 con 0x7f3a58156ac0 2024-08-17T02:45:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.162+0000 7f3a4f400640 1 -- 172.21.15.16:0/3403737063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a400145e0 con 0x7f3a58156ac0 2024-08-17T02:45:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.162+0000 7f3a4f400640 1 -- 172.21.15.16:0/3403737063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40014960 con 0x7f3a58156ac0 2024-08-17T02:45:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.162+0000 7f3a5ee00640 1 -- 172.21.15.16:0/3403737063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a58169c10 con 0x7f3a58156ac0 2024-08-17T02:45:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.162+0000 7f3a5ee00640 1 -- 172.21.15.16:0/3403737063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5814dd10 con 0x7f3a58156ac0 2024-08-17T02:45:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.163+0000 7f3a5ee00640 1 -- 172.21.15.16:0/3403737063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5814df30 con 0x7f3a58156ac0 2024-08-17T02:45:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.164+0000 7f3a4f400640 1 -- 172.21.15.16:0/3403737063 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f3a40014d10 con 0x7f3a58156ac0 2024-08-17T02:45:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.164+0000 7f3a4f400640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3a34065150 0x7f3a34067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.165+0000 7f3a4f400640 1 -- 172.21.15.16:0/3403737063 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3a400819a0 con 0x7f3a58156ac0 2024-08-17T02:45:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.165+0000 7f3a57e00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3a34065150 0x7f3a34067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.166+0000 7f3a57e00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3a34065150 0x7f3a34067600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a48034000 tx=0x7f3a48035000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.168+0000 7f3a4f400640 1 -- 172.21.15.16:0/3403737063 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f3a4004de40 con 0x7f3a58156ac0 2024-08-17T02:45:50.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.353+0000 7f3a5ee00640 1 -- 172.21.15.16:0/3403737063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a58064db0 con 0x7f3a58156ac0 2024-08-17T02:45:50.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.354+0000 7f3a4f400640 1 -- 172.21.15.16:0/3403737063 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a40019030 con 0x7f3a58156ac0 2024-08-17T02:45:50.355 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-17T02:45:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.356+0000 7f3a5ee00640 1 -- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3a34065150 msgr2=0x7f3a34067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.356+0000 7f3a5ee00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3a34065150 0x7f3a34067600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a48034000 tx=0x7f3a48035000 comp rx=0 tx=0).stop 2024-08-17T02:45:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.356+0000 7f3a5ee00640 1 -- 172.21.15.16:0/3403737063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a58156ac0 msgr2=0x7f3a580a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.356+0000 7f3a5ee00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a58156ac0 0x7f3a580a98e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3a40001eb0 tx=0x7f3a40009040 comp rx=0 tx=0).stop 2024-08-17T02:45:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.356+0000 7f3a5ee00640 1 -- 172.21.15.16:0/3403737063 shutdown_connections 2024-08-17T02:45:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.356+0000 7f3a5ee00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3a34065150 0x7f3a34067600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.356+0000 7f3a5ee00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3a58168db0 0x7f3a5814f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.356+0000 7f3a5ee00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a58156ac0 0x7f3a580a98e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.356+0000 7f3a5ee00640 1 --2- 172.21.15.16:0/3403737063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a3c003a20 0x7f3a58151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.356+0000 7f3a5ee00640 1 -- 172.21.15.16:0/3403737063 >> 172.21.15.16:0/3403737063 conn(0x7f3a58067750 msgr2=0x7f3a580687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:50.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.356+0000 7f3a5ee00640 1 -- 172.21.15.16:0/3403737063 shutdown_connections 2024-08-17T02:45:50.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.356+0000 7f3a5ee00640 1 -- 172.21.15.16:0/3403737063 wait complete. 2024-08-17T02:45:50.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:45:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.525+0000 7f65da800640 1 Processor -- start 2024-08-17T02:45:50.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.525+0000 7f65da800640 1 -- start start 2024-08-17T02:45:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.526+0000 7f65da800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65d40c8bd0 0x7f65d40c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.526+0000 7f65da800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d40c9570 con 0x7f65d40adcf0 2024-08-17T02:45:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.526+0000 7f65da800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d40ab130 con 0x7f65d4156c50 2024-08-17T02:45:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.526+0000 7f65da800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d40ae5a0 con 0x7f65d40c8bd0 2024-08-17T02:45:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.527+0000 7f65d2a00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f65d40adcf0 0x7f65d40ab5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52838/0 (socket says 172.21.15.16:52838) 2024-08-17T02:45:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.527+0000 7f65d2a00640 1 -- 172.21.15.16:0/2420862050 learned_addr learned my addr 172.21.15.16:0/2420862050 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:50.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.527+0000 7f65d3e00640 1 --2- 172.21.15.16:0/2420862050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65d40c8bd0 0x7f65d40c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.527+0000 7f65d2000640 1 -- 172.21.15.16:0/2420862050 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452903125 0 0) 0x7f65c0001110 con 0x7f65d40adcf0 2024-08-17T02:45:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.527+0000 7f65d2000640 1 -- 172.21.15.16:0/2420862050 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65b80034e0 con 0x7f65d40adcf0 2024-08-17T02:45:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.527+0000 7f65d3e00640 1 -- 172.21.15.16:0/2420862050 >> v1:172.21.15.142:6789/0 conn(0x7f65d4156c50 legacy=0x7f65d40c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.527+0000 7f65d2000640 1 -- 172.21.15.16:0/2420862050 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998950470 0 0) 0x7f65d40ab130 con 0x7f65d4156c50 2024-08-17T02:45:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.527+0000 7f65d3e00640 1 -- 172.21.15.16:0/2420862050 >> v1:172.21.15.16:6789/0 conn(0x7f65d40adcf0 legacy=0x7f65d40ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.527+0000 7f65d3e00640 1 -- 172.21.15.16:0/2420862050 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d40ae2c0 con 0x7f65d40c8bd0 2024-08-17T02:45:50.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.528+0000 7f65d3e00640 1 --2- 172.21.15.16:0/2420862050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65d40c8bd0 0x7f65d40c9030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009e90 tx=0x7f65cc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7621f1c4848931b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.529+0000 7f65d2000640 1 -- 172.21.15.16:0/2420862050 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc02aea0 con 0x7f65d40c8bd0 2024-08-17T02:45:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.529+0000 7f65d2000640 1 -- 172.21.15.16:0/2420862050 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc02d050 con 0x7f65d40c8bd0 2024-08-17T02:45:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.529+0000 7f65d2000640 1 -- 172.21.15.16:0/2420862050 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc007760 con 0x7f65d40c8bd0 2024-08-17T02:45:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.529+0000 7f65da800640 1 -- 172.21.15.16:0/2420862050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65d40c8bd0 msgr2=0x7f65d40c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.529+0000 7f65da800640 1 --2- 172.21.15.16:0/2420862050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65d40c8bd0 0x7f65d40c9030 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f65cc009e90 tx=0x7f65cc009780 comp rx=0 tx=0).stop 2024-08-17T02:45:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.529+0000 7f65da800640 1 -- 172.21.15.16:0/2420862050 shutdown_connections 2024-08-17T02:45:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.529+0000 7f65da800640 1 --2- 172.21.15.16:0/2420862050 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65d40c8bd0 0x7f65d40c9030 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.529+0000 7f65da800640 1 -- 172.21.15.16:0/2420862050 >> 172.21.15.16:0/2420862050 conn(0x7f65d4067750 msgr2=0x7f65d40c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.529+0000 7f65da800640 1 -- 172.21.15.16:0/2420862050 shutdown_connections 2024-08-17T02:45:50.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.529+0000 7f65da800640 1 -- 172.21.15.16:0/2420862050 wait complete. 2024-08-17T02:45:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.531+0000 7f65da800640 1 Processor -- start 2024-08-17T02:45:50.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.531+0000 7f65da800640 1 -- start start 2024-08-17T02:45:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65da800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d40adcf0 0x7f65d414ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65da800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65d40c8bd0 0x7f65d41595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65da800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65d4156c50 0x7f65d4150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65da800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4158ba0 con 0x7f65d40adcf0 2024-08-17T02:45:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65da800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d41518b0 con 0x7f65d4156c50 2024-08-17T02:45:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65da800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4151b90 con 0x7f65d40c8bd0 2024-08-17T02:45:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d40adcf0 0x7f65d414ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65d3400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d40adcf0 0x7f65d414ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36556/0 (socket says 172.21.15.16:36556) 2024-08-17T02:45:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65d3400640 1 -- 172.21.15.16:0/2310089729 learned_addr learned my addr 172.21.15.16:0/2310089729 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65d2a00640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65d40c8bd0 0x7f65d41595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65d3e00640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65d4156c50 0x7f65d4150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65d3400640 1 -- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65d40c8bd0 msgr2=0x7f65d41595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65d3400640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65d40c8bd0 0x7f65d41595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65d3400640 1 -- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65d4156c50 msgr2=0x7f65d4150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65d3400640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65d4156c50 0x7f65d4150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65d3400640 1 -- 172.21.15.16:0/2310089729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc009a20 con 0x7f65d40adcf0 2024-08-17T02:45:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.532+0000 7f65d2a00640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65d40c8bd0 0x7f65d41595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:50.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.533+0000 7f65d3400640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d40adcf0 0x7f65d414ef30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f65c400c5e0 tx=0x7f65c4007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.533+0000 7f65caa00640 1 -- 172.21.15.16:0/2310089729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4012630 con 0x7f65d40adcf0 2024-08-17T02:45:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.533+0000 7f65caa00640 1 -- 172.21.15.16:0/2310089729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c40127b0 con 0x7f65d40adcf0 2024-08-17T02:45:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.533+0000 7f65caa00640 1 -- 172.21.15.16:0/2310089729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4012b70 con 0x7f65d40adcf0 2024-08-17T02:45:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.533+0000 7f65da800640 1 -- 172.21.15.16:0/2310089729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d4151e70 con 0x7f65d40adcf0 2024-08-17T02:45:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.533+0000 7f65da800640 1 -- 172.21.15.16:0/2310089729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d41573d0 con 0x7f65d40adcf0 2024-08-17T02:45:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.534+0000 7f65da800640 1 -- 172.21.15.16:0/2310089729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d4071050 con 0x7f65d40adcf0 2024-08-17T02:45:50.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.535+0000 7f65caa00640 1 -- 172.21.15.16:0/2310089729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f65c401b400 con 0x7f65d40adcf0 2024-08-17T02:45:50.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.536+0000 7f65caa00640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f65b4065180 0x7f65b4067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.536+0000 7f65caa00640 1 -- 172.21.15.16:0/2310089729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f65c40820f0 con 0x7f65d40adcf0 2024-08-17T02:45:50.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.536+0000 7f65d2a00640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f65b4065180 0x7f65b4067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.537+0000 7f65d2a00640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f65b4065180 0x7f65b4067630 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f65d406d940 tx=0x7f65c0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:50.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.540+0000 7f65caa00640 1 -- 172.21.15.16:0/2310089729 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f65c4050110 con 0x7f65d40adcf0 2024-08-17T02:45:50.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.721+0000 7f65da800640 1 -- 172.21.15.16:0/2310089729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65d40c89b0 con 0x7f65d40adcf0 2024-08-17T02:45:50.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.722+0000 7f65caa00640 1 -- 172.21.15.16:0/2310089729 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65c4053730 con 0x7f65d40adcf0 2024-08-17T02:45:50.723 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:45:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.724+0000 7f65da800640 1 -- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f65b4065180 msgr2=0x7f65b4067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.724+0000 7f65da800640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f65b4065180 0x7f65b4067630 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f65d406d940 tx=0x7f65c0009040 comp rx=0 tx=0).stop 2024-08-17T02:45:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.724+0000 7f65da800640 1 -- 172.21.15.16:0/2310089729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d40adcf0 msgr2=0x7f65d414ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.724+0000 7f65da800640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d40adcf0 0x7f65d414ef30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f65c400c5e0 tx=0x7f65c4007ed0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.724+0000 7f65da800640 1 -- 172.21.15.16:0/2310089729 shutdown_connections 2024-08-17T02:45:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.724+0000 7f65da800640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f65b4065180 0x7f65b4067630 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.724+0000 7f65da800640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f65d4156c50 0x7f65d4150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.724+0000 7f65da800640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f65d40c8bd0 0x7f65d41595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.724+0000 7f65da800640 1 --2- 172.21.15.16:0/2310089729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65d40adcf0 0x7f65d414ef30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.724+0000 7f65da800640 1 -- 172.21.15.16:0/2310089729 >> 172.21.15.16:0/2310089729 conn(0x7f65d4067750 msgr2=0x7f65d40ab5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.724+0000 7f65da800640 1 -- 172.21.15.16:0/2310089729 shutdown_connections 2024-08-17T02:45:50.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.724+0000 7f65da800640 1 -- 172.21.15.16:0/2310089729 wait complete. 2024-08-17T02:45:50.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:45:50.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.892+0000 7f60fc600640 1 Processor -- start 2024-08-17T02:45:50.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.892+0000 7f60fc600640 1 -- start start 2024-08-17T02:45:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60fc600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60f40d0890 0x7f60f40b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60fc600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f60f40b0d50 0x7f60f40b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60fc600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f40b1180 con 0x7f60f415c2f0 2024-08-17T02:45:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60fc600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f40b1e10 con 0x7f60f40d0890 2024-08-17T02:45:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60fc600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f40afbb0 con 0x7f60f40b0d50 2024-08-17T02:45:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60f9c00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f60f415c2f0 0x7f60f40af3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52840/0 (socket says 172.21.15.16:52840) 2024-08-17T02:45:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60f9c00640 1 -- 172.21.15.16:0/698825516 learned_addr learned my addr 172.21.15.16:0/698825516 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60f9200640 1 --2- 172.21.15.16:0/698825516 >> v2:172.21.15.16:3300/0 conn(0x7f60f40d0890 0x7f60f40b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60fa600640 1 --2- 172.21.15.16:0/698825516 >> v2:172.21.15.142:3300/0 conn(0x7f60f40b0d50 0x7f60f40b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60f9200640 1 -- 172.21.15.16:0/698825516 >> v2:172.21.15.142:3300/0 conn(0x7f60f40b0d50 msgr2=0x7f60f40b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60f9200640 1 --2- 172.21.15.16:0/698825516 >> v2:172.21.15.142:3300/0 conn(0x7f60f40b0d50 0x7f60f40b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60f9200640 1 -- 172.21.15.16:0/698825516 >> v1:172.21.15.16:6789/0 conn(0x7f60f415c2f0 legacy=0x7f60f40af3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.895+0000 7f60f9200640 1 -- 172.21.15.16:0/698825516 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f40afe90 con 0x7f60f40d0890 2024-08-17T02:45:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.894+0000 7f60f3e00640 1 -- 172.21.15.16:0/698825516 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266133033 0 0) 0x7f60f40b1180 con 0x7f60f415c2f0 2024-08-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.896+0000 7f60f9200640 1 --2- 172.21.15.16:0/698825516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f40d0890 0x7f60f40b0810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f60f4088fd0 tx=0x7f60e00090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2e94177b403a72 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.896+0000 7f60f3e00640 1 -- 172.21.15.16:0/698825516 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e002adb0 con 0x7f60f40d0890 2024-08-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.896+0000 7f60f3e00640 1 -- 172.21.15.16:0/698825516 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e0032440 con 0x7f60f40d0890 2024-08-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.896+0000 7f60f3e00640 1 -- 172.21.15.16:0/698825516 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e00327e0 con 0x7f60f40d0890 2024-08-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.896+0000 7f60fc600640 1 -- 172.21.15.16:0/698825516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f40d0890 msgr2=0x7f60f40b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.896+0000 7f60fc600640 1 --2- 172.21.15.16:0/698825516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f40d0890 0x7f60f40b0810 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f60f4088fd0 tx=0x7f60e00090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.896+0000 7f60fc600640 1 -- 172.21.15.16:0/698825516 shutdown_connections 2024-08-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.896+0000 7f60fc600640 1 --2- 172.21.15.16:0/698825516 >> v2:172.21.15.142:3300/0 conn(0x7f60f40b0d50 0x7f60f40b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.896+0000 7f60fc600640 1 --2- 172.21.15.16:0/698825516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f40d0890 0x7f60f40b0810 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.896+0000 7f60fc600640 1 -- 172.21.15.16:0/698825516 >> 172.21.15.16:0/698825516 conn(0x7f60f40cfdb0 msgr2=0x7f60f40d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.897+0000 7f60fc600640 1 -- 172.21.15.16:0/698825516 shutdown_connections 2024-08-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.897+0000 7f60fc600640 1 -- 172.21.15.16:0/698825516 wait complete. 2024-08-17T02:45:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.898+0000 7f60fc600640 1 Processor -- start 2024-08-17T02:45:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.898+0000 7f60fc600640 1 -- start start 2024-08-17T02:45:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.899+0000 7f60fc600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60f40b0d50 0x7f60f416bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.899+0000 7f60fc600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f40d0890 0x7f60f40ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.899+0000 7f60fc600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60f415c2f0 0x7f60f40ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.899+0000 7f60fc600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40aeb30 con 0x7f60f40d0890 2024-08-17T02:45:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.899+0000 7f60fc600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40ce2b0 con 0x7f60f415c2f0 2024-08-17T02:45:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.899+0000 7f60fc600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40adf20 con 0x7f60f40b0d50 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.899+0000 7f60f9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f40d0890 0x7f60f40ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.899+0000 7f60f9200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f40d0890 0x7f60f40ad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36578/0 (socket says 172.21.15.16:36578) 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.899+0000 7f60f9200640 1 -- 172.21.15.16:0/1853456628 learned_addr learned my addr 172.21.15.16:0/1853456628 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.899+0000 7f60fa600640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60f415c2f0 0x7f60f40ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.899+0000 7f60f9c00640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60f40b0d50 0x7f60f416bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.900+0000 7f60f9200640 1 -- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60f40b0d50 msgr2=0x7f60f416bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.900+0000 7f60f9200640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60f40b0d50 0x7f60f416bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.900+0000 7f60f9200640 1 -- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60f415c2f0 msgr2=0x7f60f40ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.900+0000 7f60f9200640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60f415c2f0 0x7f60f40ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.900+0000 7f60f9200640 1 -- 172.21.15.16:0/1853456628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e0009390 con 0x7f60f40d0890 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.900+0000 7f60f9200640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f40d0890 0x7f60f40ad3b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009360 tx=0x7f60e0032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.900+0000 7f60f0c00640 1 -- 172.21.15.16:0/1853456628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e0031de0 con 0x7f60f40d0890 2024-08-17T02:45:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.900+0000 7f60f0c00640 1 -- 172.21.15.16:0/1853456628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e003d420 con 0x7f60f40d0890 2024-08-17T02:45:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.900+0000 7f60fc600640 1 -- 172.21.15.16:0/1853456628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f40ae200 con 0x7f60f40d0890 2024-08-17T02:45:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.900+0000 7f60f0c00640 1 -- 172.21.15.16:0/1853456628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e003d7e0 con 0x7f60f40d0890 2024-08-17T02:45:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.900+0000 7f60fc600640 1 -- 172.21.15.16:0/1853456628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f40ae7c0 con 0x7f60f40d0890 2024-08-17T02:45:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.901+0000 7f60fc600640 1 -- 172.21.15.16:0/1853456628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f40aece0 con 0x7f60f40d0890 2024-08-17T02:45:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.902+0000 7f60f0c00640 1 -- 172.21.15.16:0/1853456628 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f60e0039040 con 0x7f60f40d0890 2024-08-17T02:45:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.903+0000 7f60f0c00640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f60d4065150 0x7f60d4067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.903+0000 7f60f0c00640 1 -- 172.21.15.16:0/1853456628 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f60e00a2460 con 0x7f60f40d0890 2024-08-17T02:45:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.903+0000 7f60f9c00640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f60d4065150 0x7f60d4067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.904+0000 7f60f9c00640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f60d4065150 0x7f60d4067600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f60e8007160 tx=0x7f60e8007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:50.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:50.907+0000 7f60f0c00640 1 -- 172.21.15.16:0/1853456628 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f60f40aece0 con 0x7f60f40d0890 2024-08-17T02:45:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.088+0000 7f60fc600640 1 -- 172.21.15.16:0/1853456628 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f60f4071050 con 0x7f60f40d0890 2024-08-17T02:45:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.089+0000 7f60f0c00640 1 -- 172.21.15.16:0/1853456628 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f60e0070530 con 0x7f60f40d0890 2024-08-17T02:45:51.090 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-08-17T02:45:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.091+0000 7f60fc600640 1 -- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f60d4065150 msgr2=0x7f60d4067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.091+0000 7f60fc600640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f60d4065150 0x7f60d4067600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f60e8007160 tx=0x7f60e8007ec0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.091+0000 7f60fc600640 1 -- 172.21.15.16:0/1853456628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f40d0890 msgr2=0x7f60f40ad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.091+0000 7f60fc600640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f40d0890 0x7f60f40ad3b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f60e0009360 tx=0x7f60e0032000 comp rx=0 tx=0).stop 2024-08-17T02:45:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.091+0000 7f60fc600640 1 -- 172.21.15.16:0/1853456628 shutdown_connections 2024-08-17T02:45:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.091+0000 7f60fc600640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f60d4065150 0x7f60d4067600 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.091+0000 7f60fc600640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60f415c2f0 0x7f60f40ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.091+0000 7f60fc600640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60f40d0890 0x7f60f40ad3b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.091+0000 7f60fc600640 1 --2- 172.21.15.16:0/1853456628 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f60f40b0d50 0x7f60f416bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.091+0000 7f60fc600640 1 -- 172.21.15.16:0/1853456628 >> 172.21.15.16:0/1853456628 conn(0x7f60f40cfdb0 msgr2=0x7f60f41557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.091+0000 7f60fc600640 1 -- 172.21.15.16:0/1853456628 shutdown_connections 2024-08-17T02:45:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.091+0000 7f60fc600640 1 -- 172.21.15.16:0/1853456628 wait complete. 2024-08-17T02:45:51.176 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2024-08-17T02:45:51.176 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-17T02:45:51.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:51.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.327+0000 7f7228200640 1 Processor -- start 2024-08-17T02:45:51.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.327+0000 7f7228200640 1 -- start start 2024-08-17T02:45:51.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.328+0000 7f7228200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f72201401b0 0x7f72200caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.328+0000 7f7228200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7220140580 0x7f72200b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.328+0000 7f7228200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72200b1270 0x7f72200c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.328+0000 7f7228200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72200cb450 con 0x7f7220140580 2024-08-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7228200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72200b16d0 con 0x7f72201401b0 2024-08-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7228200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72200caa80 con 0x7f72200b1270 2024-08-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7224e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7220140580 0x7f72200b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7224e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7220140580 0x7f72200b0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36594/0 (socket says 172.21.15.16:36594) 2024-08-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7224e00640 1 -- 172.21.15.16:0/1252664926 learned_addr learned my addr 172.21.15.16:0/1252664926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7226200640 1 --2- 172.21.15.16:0/1252664926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72200b1270 0x7f72200c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7225800640 1 --2- 172.21.15.16:0/1252664926 >> v2:172.21.15.142:3300/0 conn(0x7f72201401b0 0x7f72200caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7224e00640 1 -- 172.21.15.16:0/1252664926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72200b1270 msgr2=0x7f72200c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7224e00640 1 --2- 172.21.15.16:0/1252664926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72200b1270 0x7f72200c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7224e00640 1 -- 172.21.15.16:0/1252664926 >> v2:172.21.15.142:3300/0 conn(0x7f72201401b0 msgr2=0x7f72200caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7224e00640 1 --2- 172.21.15.16:0/1252664926 >> v2:172.21.15.142:3300/0 conn(0x7f72201401b0 0x7f72200caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7224e00640 1 -- 172.21.15.16:0/1252664926 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72200c9e70 con 0x7f7220140580 2024-08-17T02:45:51.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.329+0000 7f7225800640 1 --2- 172.21.15.16:0/1252664926 >> v2:172.21.15.142:3300/0 conn(0x7f72201401b0 0x7f72200caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:51.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.330+0000 7f7224e00640 1 --2- 172.21.15.16:0/1252664926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7220140580 0x7f72200b0d30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7220068650 tx=0x7f720c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549273ee63a9ebca server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:51.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.330+0000 7f721fe00640 1 -- 172.21.15.16:0/1252664926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c02ad40 con 0x7f7220140580 2024-08-17T02:45:51.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.330+0000 7f721fe00640 1 -- 172.21.15.16:0/1252664926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c032440 con 0x7f7220140580 2024-08-17T02:45:51.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.330+0000 7f721fe00640 1 -- 172.21.15.16:0/1252664926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c02aea0 con 0x7f7220140580 2024-08-17T02:45:51.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.331+0000 7f7228200640 1 -- 172.21.15.16:0/1252664926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7220140580 msgr2=0x7f72200b0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.331+0000 7f7228200640 1 --2- 172.21.15.16:0/1252664926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7220140580 0x7f72200b0d30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7220068650 tx=0x7f720c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.331+0000 7f7228200640 1 -- 172.21.15.16:0/1252664926 shutdown_connections 2024-08-17T02:45:51.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.331+0000 7f7228200640 1 --2- 172.21.15.16:0/1252664926 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72200b1270 0x7f72200c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.331+0000 7f7228200640 1 --2- 172.21.15.16:0/1252664926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7220140580 0x7f72200b0d30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.331+0000 7f7228200640 1 --2- 172.21.15.16:0/1252664926 >> v2:172.21.15.142:3300/0 conn(0x7f72201401b0 0x7f72200caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.331+0000 7f7228200640 1 -- 172.21.15.16:0/1252664926 >> 172.21.15.16:0/1252664926 conn(0x7f72200cb9b0 msgr2=0x7f72200cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:51.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.331+0000 7f7228200640 1 -- 172.21.15.16:0/1252664926 shutdown_connections 2024-08-17T02:45:51.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.331+0000 7f7228200640 1 -- 172.21.15.16:0/1252664926 wait complete. 2024-08-17T02:45:51.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.333+0000 7f7228200640 1 Processor -- start 2024-08-17T02:45:51.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.333+0000 7f7228200640 1 -- start start 2024-08-17T02:45:51.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7228200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72200b1270 0x7f722015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7228200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72201401b0 0x7f722015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7228200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7220140580 0x7f722013dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7228200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722015e640 con 0x7f7220140580 2024-08-17T02:45:51.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7228200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722015e0c0 con 0x7f72200b1270 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7228200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722015e360 con 0x7f72201401b0 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7226200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7220140580 0x7f722013dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7226200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7220140580 0x7f722013dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36606/0 (socket says 172.21.15.16:36606) 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7226200640 1 -- 172.21.15.16:0/1505362233 learned_addr learned my addr 172.21.15.16:0/1505362233 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7225800640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72200b1270 0x7f722015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7224e00640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72201401b0 0x7f722015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7226200640 1 -- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72201401b0 msgr2=0x7f722015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7226200640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72201401b0 0x7f722015db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7226200640 1 -- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72200b1270 msgr2=0x7f722015b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7226200640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72200b1270 0x7f722015b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7226200640 1 -- 172.21.15.16:0/1505362233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c009450 con 0x7f7220140580 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.334+0000 7f7224e00640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72201401b0 0x7f722015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.335+0000 7f7226200640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7220140580 0x7f722013dbd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7218007f20 tx=0x7f7218009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:51.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.335+0000 7f721cc00640 1 -- 172.21.15.16:0/1505362233 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721800ee30 con 0x7f7220140580 2024-08-17T02:45:51.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.335+0000 7f721cc00640 1 -- 172.21.15.16:0/1505362233 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7218014420 con 0x7f7220140580 2024-08-17T02:45:51.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.335+0000 7f721cc00640 1 -- 172.21.15.16:0/1505362233 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72180147e0 con 0x7f7220140580 2024-08-17T02:45:51.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.335+0000 7f7228200640 1 -- 172.21.15.16:0/1505362233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722013b7c0 con 0x7f7220140580 2024-08-17T02:45:51.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.335+0000 7f7228200640 1 -- 172.21.15.16:0/1505362233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722013bd80 con 0x7f7220140580 2024-08-17T02:45:51.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.336+0000 7f7228200640 1 -- 172.21.15.16:0/1505362233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722013bfa0 con 0x7f7220140580 2024-08-17T02:45:51.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.337+0000 7f721cc00640 1 -- 172.21.15.16:0/1505362233 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f7218014b50 con 0x7f7220140580 2024-08-17T02:45:51.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.338+0000 7f721cc00640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7200065180 0x7f7200067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.338+0000 7f721cc00640 1 -- 172.21.15.16:0/1505362233 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7218082c20 con 0x7f7220140580 2024-08-17T02:45:51.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.338+0000 7f7225800640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7200065180 0x7f7200067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.339+0000 7f7225800640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7200065180 0x7f7200067630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7214006fd0 tx=0x7f7214008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:51.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.342+0000 7f721cc00640 1 -- 172.21.15.16:0/1505362233 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7218050c40 con 0x7f7220140580 2024-08-17T02:45:51.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.525+0000 7f7228200640 1 -- 172.21.15.16:0/1505362233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7220064db0 con 0x7f7220140580 2024-08-17T02:45:51.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.526+0000 7f721cc00640 1 -- 172.21.15.16:0/1505362233 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f7218054260 con 0x7f7220140580 2024-08-17T02:45:51.528 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:51.528 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.529+0000 7f7228200640 1 -- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7200065180 msgr2=0x7f7200067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.529+0000 7f7228200640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7200065180 0x7f7200067630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7214006fd0 tx=0x7f7214008040 comp rx=0 tx=0).stop 2024-08-17T02:45:51.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.529+0000 7f7228200640 1 -- 172.21.15.16:0/1505362233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7220140580 msgr2=0x7f722013dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.529+0000 7f7228200640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7220140580 0x7f722013dbd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7218007f20 tx=0x7f7218009860 comp rx=0 tx=0).stop 2024-08-17T02:45:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.529+0000 7f7228200640 1 -- 172.21.15.16:0/1505362233 shutdown_connections 2024-08-17T02:45:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.529+0000 7f7228200640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7200065180 0x7f7200067630 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.529+0000 7f7228200640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7220140580 0x7f722013dbd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.529+0000 7f7228200640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72201401b0 0x7f722015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.529+0000 7f7228200640 1 --2- 172.21.15.16:0/1505362233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72200b1270 0x7f722015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.529+0000 7f7228200640 1 -- 172.21.15.16:0/1505362233 >> 172.21.15.16:0/1505362233 conn(0x7f72200cb9b0 msgr2=0x7f72200aed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.529+0000 7f7228200640 1 -- 172.21.15.16:0/1505362233 shutdown_connections 2024-08-17T02:45:51.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.529+0000 7f7228200640 1 -- 172.21.15.16:0/1505362233 wait complete. 2024-08-17T02:45:51.546 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:45:14.291903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:45:18.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:45:18.430718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:45:51.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:45:51.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.699+0000 7fe121a00640 1 Processor -- start 2024-08-17T02:45:51.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.699+0000 7fe121a00640 1 -- start start 2024-08-17T02:45:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.701+0000 7fe121a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe11c1564b0 0x7fe11c0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.701+0000 7fe121a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe11c156880 0x7fe11c159100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.701+0000 7fe121a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11c159640 con 0x7fe11c0aec60 2024-08-17T02:45:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.701+0000 7fe121a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11c0ae7d0 con 0x7fe11c156880 2024-08-17T02:45:51.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.701+0000 7fe121a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0ae4e0 con 0x7fe11c1564b0 2024-08-17T02:45:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.701+0000 7fe11b400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe11c156880 0x7fe11c159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.701+0000 7fe11be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe11c1564b0 0x7fe11c0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.701+0000 7fe11b400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fe11c156880 0x7fe11c159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:42870/0 (socket says 172.21.15.16:42870) 2024-08-17T02:45:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.701+0000 7fe11b400640 1 -- 172.21.15.16:0/2740951671 learned_addr learned my addr 172.21.15.16:0/2740951671 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:51.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.701+0000 7fe121000640 1 --1- 172.21.15.16:0/2740951671 >> v1:172.21.15.142:6789/0 conn(0x7fe11c0aec60 0x7fe11c0a98e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:39438/0 (socket says 172.21.15.16:39438) 2024-08-17T02:45:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.702+0000 7fe11b400640 1 -- 172.21.15.16:0/2740951671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe11c1564b0 msgr2=0x7fe11c0cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.702+0000 7fe11b400640 1 --2- 172.21.15.16:0/2740951671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe11c1564b0 0x7fe11c0cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.702+0000 7fe11b400640 1 -- 172.21.15.16:0/2740951671 >> v1:172.21.15.142:6789/0 conn(0x7fe11c0aec60 legacy=0x7fe11c0a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.702+0000 7fe11b400640 1 -- 172.21.15.16:0/2740951671 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c159870 con 0x7fe11c156880 2024-08-17T02:45:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.702+0000 7fe11be00640 1 --2- 172.21.15.16:0/2740951671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe11c1564b0 0x7fe11c0cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.703+0000 7fe11b400640 1 --2- 172.21.15.16:0/2740951671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c156880 0x7fe11c159100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe108009910 tx=0x7fe1080091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b1f1ce1e2c6aedc server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.703+0000 7fe11aa00640 1 -- 172.21.15.16:0/2740951671 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10802ae40 con 0x7fe11c156880 2024-08-17T02:45:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.703+0000 7fe11aa00640 1 -- 172.21.15.16:0/2740951671 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108032440 con 0x7fe11c156880 2024-08-17T02:45:51.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.703+0000 7fe11aa00640 1 -- 172.21.15.16:0/2740951671 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108032820 con 0x7fe11c156880 2024-08-17T02:45:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.703+0000 7fe121a00640 1 -- 172.21.15.16:0/2740951671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c156880 msgr2=0x7fe11c159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.703+0000 7fe121a00640 1 --2- 172.21.15.16:0/2740951671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c156880 0x7fe11c159100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe108009910 tx=0x7fe1080091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.704+0000 7fe121a00640 1 -- 172.21.15.16:0/2740951671 shutdown_connections 2024-08-17T02:45:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.704+0000 7fe121a00640 1 --2- 172.21.15.16:0/2740951671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c156880 0x7fe11c159100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.704+0000 7fe121a00640 1 --2- 172.21.15.16:0/2740951671 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe11c1564b0 0x7fe11c0cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.704+0000 7fe121a00640 1 -- 172.21.15.16:0/2740951671 >> 172.21.15.16:0/2740951671 conn(0x7fe11c068860 msgr2=0x7fe11c0cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.704+0000 7fe121a00640 1 -- 172.21.15.16:0/2740951671 shutdown_connections 2024-08-17T02:45:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.704+0000 7fe121a00640 1 -- 172.21.15.16:0/2740951671 wait complete. 2024-08-17T02:45:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.705+0000 7fe121a00640 1 Processor -- start 2024-08-17T02:45:51.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.705+0000 7fe121a00640 1 -- start start 2024-08-17T02:45:51.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.706+0000 7fe121a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c0aec60 0x7fe11c145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.706+0000 7fe121a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe11c1564b0 0x7fe11c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.706+0000 7fe121a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe11c156880 0x7fe11c06ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.706+0000 7fe121a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0b2220 con 0x7fe11c1564b0 2024-08-17T02:45:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.706+0000 7fe121a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0b1610 con 0x7fe11c0aec60 2024-08-17T02:45:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.706+0000 7fe121a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0b18f0 con 0x7fe11c156880 2024-08-17T02:45:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.706+0000 7fe11b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe11c1564b0 0x7fe11c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.706+0000 7fe11b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe11c1564b0 0x7fe11c06a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36618/0 (socket says 172.21.15.16:36618) 2024-08-17T02:45:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.706+0000 7fe11b400640 1 -- 172.21.15.16:0/328328521 learned_addr learned my addr 172.21.15.16:0/328328521 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.706+0000 7fe11be00640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c0aec60 0x7fe11c145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.706+0000 7fe121000640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe11c156880 0x7fe11c06ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.707+0000 7fe11b400640 1 -- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe11c156880 msgr2=0x7fe11c06ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.707+0000 7fe11b400640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe11c156880 0x7fe11c06ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.707+0000 7fe11b400640 1 -- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c0aec60 msgr2=0x7fe11c145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.707+0000 7fe11b400640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c0aec60 0x7fe11c145290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.707+0000 7fe11b400640 1 -- 172.21.15.16:0/328328521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c009390 con 0x7fe11c1564b0 2024-08-17T02:45:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.707+0000 7fe11be00640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c0aec60 0x7fe11c145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.707+0000 7fe11b400640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe11c1564b0 0x7fe11c06a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe108035040 tx=0x7fe108032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.707+0000 7fe113400640 1 -- 172.21.15.16:0/328328521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108031c80 con 0x7fe11c1564b0 2024-08-17T02:45:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.707+0000 7fe113400640 1 -- 172.21.15.16:0/328328521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108031e40 con 0x7fe11c1564b0 2024-08-17T02:45:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.707+0000 7fe113400640 1 -- 172.21.15.16:0/328328521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10803d670 con 0x7fe11c1564b0 2024-08-17T02:45:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.707+0000 7fe121a00640 1 -- 172.21.15.16:0/328328521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe108009450 con 0x7fe11c1564b0 2024-08-17T02:45:51.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.708+0000 7fe121a00640 1 -- 172.21.15.16:0/328328521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11c0b1eb0 con 0x7fe11c1564b0 2024-08-17T02:45:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.709+0000 7fe121a00640 1 -- 172.21.15.16:0/328328521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11c0b23d0 con 0x7fe11c1564b0 2024-08-17T02:45:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.710+0000 7fe113400640 1 -- 172.21.15.16:0/328328521 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fe108039040 con 0x7fe11c1564b0 2024-08-17T02:45:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.710+0000 7fe113400640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe0fc065150 0x7fe0fc067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:51.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.711+0000 7fe113400640 1 -- 172.21.15.16:0/328328521 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe10802ae40 con 0x7fe11c1564b0 2024-08-17T02:45:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.711+0000 7fe11be00640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe0fc065150 0x7fe0fc067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:51.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.712+0000 7fe11be00640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe0fc065150 0x7fe0fc067600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe10c011580 tx=0x7fe10c0090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:51.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.715+0000 7fe113400640 1 -- 172.21.15.16:0/328328521 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe11c0b23d0 con 0x7fe11c1564b0 2024-08-17T02:45:51.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.896+0000 7fe121a00640 1 -- 172.21.15.16:0/328328521 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe11c071050 con 0x7fe11c1564b0 2024-08-17T02:45:51.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.897+0000 7fe113400640 1 -- 172.21.15.16:0/328328521 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1080703d0 con 0x7fe11c1564b0 2024-08-17T02:45:51.898 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-17T02:45:51.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.899+0000 7fe121a00640 1 -- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe0fc065150 msgr2=0x7fe0fc067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.899+0000 7fe121a00640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe0fc065150 0x7fe0fc067600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe10c011580 tx=0x7fe10c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.899+0000 7fe121a00640 1 -- 172.21.15.16:0/328328521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe11c1564b0 msgr2=0x7fe11c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.899+0000 7fe121a00640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe11c1564b0 0x7fe11c06a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe108035040 tx=0x7fe108032000 comp rx=0 tx=0).stop 2024-08-17T02:45:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.899+0000 7fe121a00640 1 -- 172.21.15.16:0/328328521 shutdown_connections 2024-08-17T02:45:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.899+0000 7fe121a00640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe0fc065150 0x7fe0fc067600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.899+0000 7fe121a00640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe11c156880 0x7fe11c06ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.899+0000 7fe121a00640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe11c1564b0 0x7fe11c06a7a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.899+0000 7fe121a00640 1 --2- 172.21.15.16:0/328328521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c0aec60 0x7fe11c145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.899+0000 7fe121a00640 1 -- 172.21.15.16:0/328328521 >> 172.21.15.16:0/328328521 conn(0x7fe11c068860 msgr2=0x7fe11c067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.899+0000 7fe121a00640 1 -- 172.21.15.16:0/328328521 shutdown_connections 2024-08-17T02:45:51.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:51.899+0000 7fe121a00640 1 -- 172.21.15.16:0/328328521 wait complete. 2024-08-17T02:45:51.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:45:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.070+0000 7faa2d000640 1 Processor -- start 2024-08-17T02:45:52.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.071+0000 7faa2d000640 1 -- start start 2024-08-17T02:45:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa2d000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7faa28158d10 0x7faa28066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa2d000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa28159190 0x7faa28067210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa2d000640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2806e660 con 0x7faa28158940 2024-08-17T02:45:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa2d000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa280a9b70 con 0x7faa28159190 2024-08-17T02:45:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa2d000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2806da50 con 0x7faa28158d10 2024-08-17T02:45:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa27e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa28159190 0x7faa28067210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa27400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faa28158940 0x7faa2815d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52848/0 (socket says 172.21.15.16:52848) 2024-08-17T02:45:52.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa27400640 1 -- 172.21.15.16:0/918351639 learned_addr learned my addr 172.21.15.16:0/918351639 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa26a00640 1 --2- 172.21.15.16:0/918351639 >> v2:172.21.15.142:3300/0 conn(0x7faa28158d10 0x7faa28066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa27e00640 1 -- 172.21.15.16:0/918351639 >> v2:172.21.15.142:3300/0 conn(0x7faa28158d10 msgr2=0x7faa28066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa27e00640 1 --2- 172.21.15.16:0/918351639 >> v2:172.21.15.142:3300/0 conn(0x7faa28158d10 0x7faa28066cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa27e00640 1 -- 172.21.15.16:0/918351639 >> v1:172.21.15.16:6789/0 conn(0x7faa28158940 legacy=0x7faa2815d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.072+0000 7faa27e00640 1 -- 172.21.15.16:0/918351639 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2806dd30 con 0x7faa28159190 2024-08-17T02:45:52.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.073+0000 7faa27e00640 1 --2- 172.21.15.16:0/918351639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28159190 0x7faa28067210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faa2806d190 tx=0x7faa20009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ba9105802683b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.074+0000 7faa26000640 1 -- 172.21.15.16:0/918351639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20007550 con 0x7faa28159190 2024-08-17T02:45:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.074+0000 7faa26000640 1 -- 172.21.15.16:0/918351639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa2002d050 con 0x7faa28159190 2024-08-17T02:45:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.074+0000 7faa26000640 1 -- 172.21.15.16:0/918351639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20007920 con 0x7faa28159190 2024-08-17T02:45:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.074+0000 7faa2d000640 1 -- 172.21.15.16:0/918351639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28159190 msgr2=0x7faa28067210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.074+0000 7faa2d000640 1 --2- 172.21.15.16:0/918351639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28159190 0x7faa28067210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faa2806d190 tx=0x7faa20009780 comp rx=0 tx=0).stop 2024-08-17T02:45:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.074+0000 7faa2d000640 1 -- 172.21.15.16:0/918351639 shutdown_connections 2024-08-17T02:45:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.074+0000 7faa2d000640 1 --2- 172.21.15.16:0/918351639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28159190 0x7faa28067210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.074+0000 7faa2d000640 1 --2- 172.21.15.16:0/918351639 >> v2:172.21.15.142:3300/0 conn(0x7faa28158d10 0x7faa28066cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.074+0000 7faa2d000640 1 -- 172.21.15.16:0/918351639 >> 172.21.15.16:0/918351639 conn(0x7faa28067de0 msgr2=0x7faa280681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.074+0000 7faa2d000640 1 -- 172.21.15.16:0/918351639 shutdown_connections 2024-08-17T02:45:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.075+0000 7faa2d000640 1 -- 172.21.15.16:0/918351639 wait complete. 2024-08-17T02:45:52.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.076+0000 7faa2d000640 1 Processor -- start 2024-08-17T02:45:52.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.076+0000 7faa2d000640 1 -- start start 2024-08-17T02:45:52.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.077+0000 7faa2d000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa28158940 0x7faa28153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.077+0000 7faa2d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa28158d10 0x7faa28154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.077+0000 7faa2d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28159190 0x7faa281551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.077+0000 7faa2d000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa28169cc0 con 0x7faa28159190 2024-08-17T02:45:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.077+0000 7faa2d000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa28169fa0 con 0x7faa28158d10 2024-08-17T02:45:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.077+0000 7faa2d000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2816a280 con 0x7faa28158940 2024-08-17T02:45:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.077+0000 7faa27e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28159190 0x7faa281551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.077+0000 7faa27e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28159190 0x7faa281551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36630/0 (socket says 172.21.15.16:36630) 2024-08-17T02:45:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.077+0000 7faa27e00640 1 -- 172.21.15.16:0/3309916365 learned_addr learned my addr 172.21.15.16:0/3309916365 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.077+0000 7faa27400640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa28158940 0x7faa28153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.077+0000 7faa26a00640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa28158d10 0x7faa28154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.078+0000 7faa27e00640 1 -- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa28158940 msgr2=0x7faa28153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.078+0000 7faa27e00640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa28158940 0x7faa28153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.078+0000 7faa27e00640 1 -- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa28158d10 msgr2=0x7faa28154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.078+0000 7faa27e00640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa28158d10 0x7faa28154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.078+0000 7faa27e00640 1 -- 172.21.15.16:0/3309916365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa20009a20 con 0x7faa28159190 2024-08-17T02:45:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.078+0000 7faa27e00640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28159190 0x7faa281551f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faa200099f0 tx=0x7faa20037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.078+0000 7faa1ea00640 1 -- 172.21.15.16:0/3309916365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20031b70 con 0x7faa28159190 2024-08-17T02:45:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.078+0000 7faa1ea00640 1 -- 172.21.15.16:0/3309916365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa20031cd0 con 0x7faa28159190 2024-08-17T02:45:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.078+0000 7faa1ea00640 1 -- 172.21.15.16:0/3309916365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa20045690 con 0x7faa28159190 2024-08-17T02:45:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.078+0000 7faa2d000640 1 -- 172.21.15.16:0/3309916365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2816a560 con 0x7faa28159190 2024-08-17T02:45:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.079+0000 7faa2d000640 1 -- 172.21.15.16:0/3309916365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa28153a70 con 0x7faa28159190 2024-08-17T02:45:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.079+0000 7faa2d000640 1 -- 172.21.15.16:0/3309916365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2816aa60 con 0x7faa28159190 2024-08-17T02:45:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.080+0000 7faa1ea00640 1 -- 172.21.15.16:0/3309916365 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7faa2003c420 con 0x7faa28159190 2024-08-17T02:45:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.081+0000 7faa1ea00640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7faa08065150 0x7faa08067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.081+0000 7faa1ea00640 1 -- 172.21.15.16:0/3309916365 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa200a3ab0 con 0x7faa28159190 2024-08-17T02:45:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.082+0000 7faa27400640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7faa08065150 0x7faa08067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.082+0000 7faa27400640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7faa08065150 0x7faa08067600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faa18007880 tx=0x7faa18007e20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:52.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.085+0000 7faa1ea00640 1 -- 172.21.15.16:0/3309916365 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7faa20071b80 con 0x7faa28159190 2024-08-17T02:45:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.266+0000 7faa2d000640 1 -- 172.21.15.16:0/3309916365 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa28071050 con 0x7faa28159190 2024-08-17T02:45:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.266+0000 7faa1ea00640 1 -- 172.21.15.16:0/3309916365 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7faa20040020 con 0x7faa28159190 2024-08-17T02:45:52.268 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:45:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.268+0000 7faa2d000640 1 -- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7faa08065150 msgr2=0x7faa08067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.268+0000 7faa2d000640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7faa08065150 0x7faa08067600 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faa18007880 tx=0x7faa18007e20 comp rx=0 tx=0).stop 2024-08-17T02:45:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.268+0000 7faa2d000640 1 -- 172.21.15.16:0/3309916365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28159190 msgr2=0x7faa281551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.268+0000 7faa2d000640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28159190 0x7faa281551f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faa200099f0 tx=0x7faa20037000 comp rx=0 tx=0).stop 2024-08-17T02:45:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.268+0000 7faa2d000640 1 -- 172.21.15.16:0/3309916365 shutdown_connections 2024-08-17T02:45:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.268+0000 7faa2d000640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7faa08065150 0x7faa08067600 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.268+0000 7faa2d000640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa28159190 0x7faa281551f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.268+0000 7faa2d000640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa28158d10 0x7faa28154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.268+0000 7faa2d000640 1 --2- 172.21.15.16:0/3309916365 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faa28158940 0x7faa28153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.268+0000 7faa2d000640 1 -- 172.21.15.16:0/3309916365 >> 172.21.15.16:0/3309916365 conn(0x7faa28067de0 msgr2=0x7faa2806a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:52.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.268+0000 7faa2d000640 1 -- 172.21.15.16:0/3309916365 shutdown_connections 2024-08-17T02:45:52.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.269+0000 7faa2d000640 1 -- 172.21.15.16:0/3309916365 wait complete. 2024-08-17T02:45:52.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:45:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.439+0000 7f514ae00640 1 Processor -- start 2024-08-17T02:45:52.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.439+0000 7f514ae00640 1 -- start start 2024-08-17T02:45:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.440+0000 7f514ae00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f51440cb360 0x7f51440cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.440+0000 7f514ae00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51440cf300 con 0x7f514415c2f0 2024-08-17T02:45:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.440+0000 7f514ae00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51440cf550 con 0x7f51440d09a0 2024-08-17T02:45:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.440+0000 7f514ae00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51440cf830 con 0x7f51440cb360 2024-08-17T02:45:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.440+0000 7f5149a00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f514415c2f0 0x7f51440cd8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52850/0 (socket says 172.21.15.16:52850) 2024-08-17T02:45:52.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.440+0000 7f5149a00640 1 -- 172.21.15.16:0/349962851 learned_addr learned my addr 172.21.15.16:0/349962851 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:52.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.440+0000 7f5149000640 1 --2- 172.21.15.16:0/349962851 >> v2:172.21.15.142:3300/0 conn(0x7f51440cb360 0x7f51440cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.441+0000 7f5143e00640 1 -- 172.21.15.16:0/349962851 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852921581 0 0) 0x7f51440cf300 con 0x7f514415c2f0 2024-08-17T02:45:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.441+0000 7f5143e00640 1 -- 172.21.15.16:0/349962851 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f512c0034e0 con 0x7f514415c2f0 2024-08-17T02:45:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.441+0000 7f5143e00640 1 -- 172.21.15.16:0/349962851 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980302035 0 0) 0x7f51440cf550 con 0x7f51440d09a0 2024-08-17T02:45:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.441+0000 7f5143e00640 1 -- 172.21.15.16:0/349962851 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51440cf300 con 0x7f51440d09a0 2024-08-17T02:45:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.441+0000 7f5149000640 1 -- 172.21.15.16:0/349962851 >> v1:172.21.15.142:6789/0 conn(0x7f51440d09a0 legacy=0x7f51440ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.441+0000 7f5149000640 1 -- 172.21.15.16:0/349962851 >> v1:172.21.15.16:6789/0 conn(0x7f514415c2f0 legacy=0x7f51440cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.441+0000 7f5149000640 1 -- 172.21.15.16:0/349962851 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440aceb0 con 0x7f51440cb360 2024-08-17T02:45:52.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.442+0000 7f5149000640 1 --2- 172.21.15.16:0/349962851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51440cb360 0x7f51440cdff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5130009850 tx=0x7f51300090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a492cfebb2d85293 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.442+0000 7f5143e00640 1 -- 172.21.15.16:0/349962851 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513002ad90 con 0x7f51440cb360 2024-08-17T02:45:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.442+0000 7f5143e00640 1 -- 172.21.15.16:0/349962851 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5130032440 con 0x7f51440cb360 2024-08-17T02:45:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.442+0000 7f5143e00640 1 -- 172.21.15.16:0/349962851 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51300327e0 con 0x7f51440cb360 2024-08-17T02:45:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.443+0000 7f514ae00640 1 -- 172.21.15.16:0/349962851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51440cb360 msgr2=0x7f51440cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.443+0000 7f514ae00640 1 --2- 172.21.15.16:0/349962851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51440cb360 0x7f51440cdff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5130009850 tx=0x7f51300090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.443+0000 7f514ae00640 1 -- 172.21.15.16:0/349962851 shutdown_connections 2024-08-17T02:45:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.443+0000 7f514ae00640 1 --2- 172.21.15.16:0/349962851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51440cb360 0x7f51440cdff0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.443+0000 7f514ae00640 1 -- 172.21.15.16:0/349962851 >> 172.21.15.16:0/349962851 conn(0x7f51440cfdb0 msgr2=0x7f51440d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.443+0000 7f514ae00640 1 -- 172.21.15.16:0/349962851 shutdown_connections 2024-08-17T02:45:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.443+0000 7f514ae00640 1 -- 172.21.15.16:0/349962851 wait complete. 2024-08-17T02:45:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.444+0000 7f514ae00640 1 Processor -- start 2024-08-17T02:45:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.444+0000 7f514ae00640 1 -- start start 2024-08-17T02:45:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.445+0000 7f514ae00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51440cb360 0x7f51441694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.445+0000 7f514ae00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51440d09a0 0x7f514416a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.445+0000 7f514ae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f514415c2f0 0x7f51440ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.445+0000 7f514ae00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440b3ed0 con 0x7f51440d09a0 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.445+0000 7f514ae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440b41b0 con 0x7f514415c2f0 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.445+0000 7f514ae00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5144169cf0 con 0x7f51440cb360 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.445+0000 7f5149000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51440d09a0 0x7f514416a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.445+0000 7f5149000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51440d09a0 0x7f514416a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36632/0 (socket says 172.21.15.16:36632) 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.445+0000 7f5149000640 1 -- 172.21.15.16:0/4264257226 learned_addr learned my addr 172.21.15.16:0/4264257226 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.446+0000 7f5149a00640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51440cb360 0x7f51441694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.446+0000 7f514a400640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f514415c2f0 0x7f51440ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.446+0000 7f5149000640 1 -- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51440cb360 msgr2=0x7f51441694c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.446+0000 7f5149000640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51440cb360 0x7f51441694c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.446+0000 7f5149000640 1 -- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f514415c2f0 msgr2=0x7f51440ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.446+0000 7f5149000640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f514415c2f0 0x7f51440ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.446+0000 7f5149000640 1 -- 172.21.15.16:0/4264257226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130009390 con 0x7f51440d09a0 2024-08-17T02:45:52.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.446+0000 7f5149000640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51440d09a0 0x7f514416a690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5130009720 tx=0x7f5130032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.446+0000 7f5140c00640 1 -- 172.21.15.16:0/4264257226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5130000c40 con 0x7f51440d09a0 2024-08-17T02:45:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.447+0000 7f5140c00640 1 -- 172.21.15.16:0/4264257226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5130031a60 con 0x7f51440d09a0 2024-08-17T02:45:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.447+0000 7f5140c00640 1 -- 172.21.15.16:0/4264257226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5130031de0 con 0x7f51440d09a0 2024-08-17T02:45:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.447+0000 7f514ae00640 1 -- 172.21.15.16:0/4264257226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5144169fd0 con 0x7f51440d09a0 2024-08-17T02:45:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.447+0000 7f514ae00640 1 -- 172.21.15.16:0/4264257226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5144169a00 con 0x7f51440d09a0 2024-08-17T02:45:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.448+0000 7f514ae00640 1 -- 172.21.15.16:0/4264257226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51440b4320 con 0x7f51440d09a0 2024-08-17T02:45:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.449+0000 7f5140c00640 1 -- 172.21.15.16:0/4264257226 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f5130039040 con 0x7f51440d09a0 2024-08-17T02:45:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.449+0000 7f5140c00640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5124065180 0x7f5124067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.449+0000 7f5140c00640 1 -- 172.21.15.16:0/4264257226 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f51300a23d0 con 0x7f51440d09a0 2024-08-17T02:45:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.450+0000 7f5149a00640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5124065180 0x7f5124067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.451+0000 7f5149a00640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5124065180 0x7f5124067630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5138005840 tx=0x7f5138009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:52.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.453+0000 7f5140c00640 1 -- 172.21.15.16:0/4264257226 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f51300704a0 con 0x7f51440d09a0 2024-08-17T02:45:52.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.637+0000 7f514ae00640 1 -- 172.21.15.16:0/4264257226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5144071050 con 0x7f51440d09a0 2024-08-17T02:45:52.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.637+0000 7f5140c00640 1 -- 172.21.15.16:0/4264257226 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5130073ac0 con 0x7f51440d09a0 2024-08-17T02:45:52.639 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-08-17T02:45:52.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.639+0000 7f514ae00640 1 -- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5124065180 msgr2=0x7f5124067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.639+0000 7f514ae00640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5124065180 0x7f5124067630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5138005840 tx=0x7f5138009040 comp rx=0 tx=0).stop 2024-08-17T02:45:52.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.639+0000 7f514ae00640 1 -- 172.21.15.16:0/4264257226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51440d09a0 msgr2=0x7f514416a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.639+0000 7f514ae00640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51440d09a0 0x7f514416a690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5130009720 tx=0x7f5130032000 comp rx=0 tx=0).stop 2024-08-17T02:45:52.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.639+0000 7f514ae00640 1 -- 172.21.15.16:0/4264257226 shutdown_connections 2024-08-17T02:45:52.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.639+0000 7f514ae00640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5124065180 0x7f5124067630 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.639+0000 7f514ae00640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f514415c2f0 0x7f51440ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.639+0000 7f514ae00640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51440d09a0 0x7f514416a690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.639+0000 7f514ae00640 1 --2- 172.21.15.16:0/4264257226 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51440cb360 0x7f51441694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.639+0000 7f514ae00640 1 -- 172.21.15.16:0/4264257226 >> 172.21.15.16:0/4264257226 conn(0x7f51440cfdb0 msgr2=0x7f5144067b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:52.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.640+0000 7f514ae00640 1 -- 172.21.15.16:0/4264257226 shutdown_connections 2024-08-17T02:45:52.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.640+0000 7f514ae00640 1 -- 172.21.15.16:0/4264257226 wait complete. 2024-08-17T02:45:52.656 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2024-08-17T02:45:52.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.806+0000 7fb961600640 1 Processor -- start 2024-08-17T02:45:52.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.806+0000 7fb961600640 1 -- start start 2024-08-17T02:45:52.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.808+0000 7fb961600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb95c155a20 0x7fb95c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.808+0000 7fb961600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb95c06df00 0x7fb95c0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.808+0000 7fb961600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c156670 con 0x7fb95c155df0 2024-08-17T02:45:52.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.808+0000 7fb961600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c0b0350 con 0x7fb95c155a20 2024-08-17T02:45:52.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.808+0000 7fb961600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c06e5e0 con 0x7fb95c06df00 2024-08-17T02:45:52.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.808+0000 7fb952000640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb95c155df0 0x7fb95c06d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52864/0 (socket says 172.21.15.16:52864) 2024-08-17T02:45:52.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.808+0000 7fb952000640 1 -- 172.21.15.16:0/1732661572 learned_addr learned my addr 172.21.15.16:0/1732661572 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:52.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.808+0000 7fb95aa00640 1 --2- 172.21.15.16:0/1732661572 >> v2:172.21.15.142:3300/0 conn(0x7fb95c155a20 0x7fb95c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.808+0000 7fb95b400640 1 --2- 172.21.15.16:0/1732661572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb95c06df00 0x7fb95c0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.809+0000 7fb95a000640 1 -- 172.21.15.16:0/1732661572 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393240708 0 0) 0x7fb95c156670 con 0x7fb95c155df0 2024-08-17T02:45:52.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.809+0000 7fb95aa00640 1 -- 172.21.15.16:0/1732661572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb95c06df00 msgr2=0x7fb95c0ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:52.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.809+0000 7fb95aa00640 1 --2- 172.21.15.16:0/1732661572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb95c06df00 0x7fb95c0ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.809+0000 7fb95aa00640 1 -- 172.21.15.16:0/1732661572 >> v1:172.21.15.16:6789/0 conn(0x7fb95c155df0 legacy=0x7fb95c06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.809+0000 7fb95aa00640 1 -- 172.21.15.16:0/1732661572 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c06e8c0 con 0x7fb95c155a20 2024-08-17T02:45:52.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.810+0000 7fb95aa00640 1 --2- 172.21.15.16:0/1732661572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb95c155a20 0x7fb95c0adf10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009850 tx=0x7fb94c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e209809a33ada83 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:52.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.810+0000 7fb95a000640 1 -- 172.21.15.16:0/1732661572 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c02ad80 con 0x7fb95c155a20 2024-08-17T02:45:52.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.810+0000 7fb95a000640 1 -- 172.21.15.16:0/1732661572 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c032440 con 0x7fb95c155a20 2024-08-17T02:45:52.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.810+0000 7fb95a000640 1 -- 172.21.15.16:0/1732661572 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0327e0 con 0x7fb95c155a20 2024-08-17T02:45:52.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.811+0000 7fb961600640 1 -- 172.21.15.16:0/1732661572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb95c155a20 msgr2=0x7fb95c0adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.811+0000 7fb961600640 1 --2- 172.21.15.16:0/1732661572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb95c155a20 0x7fb95c0adf10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009850 tx=0x7fb94c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.811+0000 7fb961600640 1 -- 172.21.15.16:0/1732661572 shutdown_connections 2024-08-17T02:45:52.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.811+0000 7fb961600640 1 --2- 172.21.15.16:0/1732661572 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb95c06df00 0x7fb95c0ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.811+0000 7fb961600640 1 --2- 172.21.15.16:0/1732661572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb95c155a20 0x7fb95c0adf10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.811+0000 7fb961600640 1 -- 172.21.15.16:0/1732661572 >> 172.21.15.16:0/1732661572 conn(0x7fb95c0cb9b0 msgr2=0x7fb95c0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:52.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.811+0000 7fb961600640 1 -- 172.21.15.16:0/1732661572 shutdown_connections 2024-08-17T02:45:52.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.811+0000 7fb961600640 1 -- 172.21.15.16:0/1732661572 wait complete. 2024-08-17T02:45:52.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.812+0000 7fb961600640 1 Processor -- start 2024-08-17T02:45:52.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.813+0000 7fb961600640 1 -- start start 2024-08-17T02:45:52.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb961600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c06df00 0x7fb95c1504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb961600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb95c155a20 0x7fb95c15d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb961600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb95c155df0 0x7fb95c0aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb961600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0a9cf0 con 0x7fb95c06df00 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb961600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0a9fd0 con 0x7fb95c155df0 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb961600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c0aa2b0 con 0x7fb95c155a20 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb95aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c06df00 0x7fb95c1504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb95aa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c06df00 0x7fb95c1504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36646/0 (socket says 172.21.15.16:36646) 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb95aa00640 1 -- 172.21.15.16:0/68575453 learned_addr learned my addr 172.21.15.16:0/68575453 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb952000640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb95c155a20 0x7fb95c15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb95b400640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb95c155df0 0x7fb95c0aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb95aa00640 1 -- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb95c155a20 msgr2=0x7fb95c15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb95aa00640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb95c155a20 0x7fb95c15d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb95aa00640 1 -- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb95c155df0 msgr2=0x7fb95c0aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:52.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.814+0000 7fb95aa00640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb95c155df0 0x7fb95c0aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:52.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.815+0000 7fb95aa00640 1 -- 172.21.15.16:0/68575453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c009390 con 0x7fb95c06df00 2024-08-17T02:45:52.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.815+0000 7fb95aa00640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c06df00 0x7fb95c1504c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009360 tx=0x7fb94c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:52.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.815+0000 7fb952a00640 1 -- 172.21.15.16:0/68575453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c000d10 con 0x7fb95c06df00 2024-08-17T02:45:52.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.815+0000 7fb952a00640 1 -- 172.21.15.16:0/68575453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c031a60 con 0x7fb95c06df00 2024-08-17T02:45:52.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.815+0000 7fb961600640 1 -- 172.21.15.16:0/68575453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95c0aa590 con 0x7fb95c06df00 2024-08-17T02:45:52.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.815+0000 7fb952a00640 1 -- 172.21.15.16:0/68575453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c031e40 con 0x7fb95c06df00 2024-08-17T02:45:52.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.815+0000 7fb961600640 1 -- 172.21.15.16:0/68575453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c150a00 con 0x7fb95c06df00 2024-08-17T02:45:52.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.816+0000 7fb961600640 1 -- 172.21.15.16:0/68575453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95c064db0 con 0x7fb95c06df00 2024-08-17T02:45:52.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.817+0000 7fb952a00640 1 -- 172.21.15.16:0/68575453 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fb94c03c400 con 0x7fb95c06df00 2024-08-17T02:45:52.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.818+0000 7fb952a00640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb938065150 0x7fb938067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:52.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.818+0000 7fb952a00640 1 -- 172.21.15.16:0/68575453 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb94c0a2850 con 0x7fb95c06df00 2024-08-17T02:45:52.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.819+0000 7fb952000640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb938065150 0x7fb938067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:52.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.819+0000 7fb952000640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb938065150 0x7fb938067600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb95c16c500 tx=0x7fb944007ec0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:52.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:52.822+0000 7fb952a00640 1 -- 172.21.15.16:0/68575453 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb94c070920 con 0x7fb95c06df00 2024-08-17T02:45:53.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.068+0000 7fb961600640 1 -- 172.21.15.16:0/68575453 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7fb95c154b60 con 0x7fb95c06df00 2024-08-17T02:45:53.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.366+0000 7fb952a00640 1 -- 172.21.15.16:0/68575453 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb94c073f40 con 0x7fb95c06df00 2024-08-17T02:45:53.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.368+0000 7fb961600640 1 -- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb938065150 msgr2=0x7fb938067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:53.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.368+0000 7fb961600640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb938065150 0x7fb938067600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb95c16c500 tx=0x7fb944007ec0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.368+0000 7fb961600640 1 -- 172.21.15.16:0/68575453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c06df00 msgr2=0x7fb95c1504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.368+0000 7fb961600640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c06df00 0x7fb95c1504c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009360 tx=0x7fb94c032000 comp rx=0 tx=0).stop 2024-08-17T02:45:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.368+0000 7fb961600640 1 -- 172.21.15.16:0/68575453 shutdown_connections 2024-08-17T02:45:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.368+0000 7fb961600640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb938065150 0x7fb938067600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.368+0000 7fb961600640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb95c155df0 0x7fb95c0aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.368+0000 7fb961600640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb95c155a20 0x7fb95c15d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.368+0000 7fb961600640 1 --2- 172.21.15.16:0/68575453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95c06df00 0x7fb95c1504c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.368+0000 7fb961600640 1 -- 172.21.15.16:0/68575453 >> 172.21.15.16:0/68575453 conn(0x7fb95c0cb9b0 msgr2=0x7fb95c158510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.368+0000 7fb961600640 1 -- 172.21.15.16:0/68575453 shutdown_connections 2024-08-17T02:45:53.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.368+0000 7fb961600640 1 -- 172.21.15.16:0/68575453 wait complete. 2024-08-17T02:45:53.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-08-17T02:45:53.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.539+0000 7f19e9000640 1 Processor -- start 2024-08-17T02:45:53.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.539+0000 7f19e9000640 1 -- start start 2024-08-17T02:45:53.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.540+0000 7f19e9000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19e40c7db0 0x7f19e40c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:53.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.541+0000 7f19e9000640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e41598f0 con 0x7f19e4155a80 2024-08-17T02:45:53.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.541+0000 7f19e9000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e4159bd0 con 0x7f19e4155e50 2024-08-17T02:45:53.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.541+0000 7f19e9000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e4159eb0 con 0x7f19e40c7db0 2024-08-17T02:45:53.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.541+0000 7f19dbe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19e40c7db0 0x7f19e40c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:53.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.541+0000 7f19e1e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f19e4155a80 0x7f19e406dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52872/0 (socket says 172.21.15.16:52872) 2024-08-17T02:45:53.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.541+0000 7f19e1e00640 1 -- 172.21.15.16:0/2642578687 learned_addr learned my addr 172.21.15.16:0/2642578687 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:53.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.541+0000 7f19dbe00640 1 -- 172.21.15.16:0/2642578687 >> v1:172.21.15.142:6789/0 conn(0x7f19e4155e50 legacy=0x7f19e40aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:53.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.541+0000 7f19dbe00640 1 -- 172.21.15.16:0/2642578687 >> v1:172.21.15.16:6789/0 conn(0x7f19e4155a80 legacy=0x7f19e406dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:53.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.541+0000 7f19dbe00640 1 -- 172.21.15.16:0/2642578687 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e415a190 con 0x7f19e40c7db0 2024-08-17T02:45:53.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.541+0000 7f19e1400640 1 -- 172.21.15.16:0/2642578687 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393367134 0 0) 0x7f19e41598f0 con 0x7f19e4155a80 2024-08-17T02:45:53.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.542+0000 7f19dbe00640 1 --2- 172.21.15.16:0/2642578687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e40c7db0 0x7f19e40c8bd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009850 tx=0x7f19cc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23247e42d5292e11 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:53.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.543+0000 7f19e1400640 1 -- 172.21.15.16:0/2642578687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc02ae40 con 0x7f19e40c7db0 2024-08-17T02:45:53.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.543+0000 7f19e1400640 1 -- 172.21.15.16:0/2642578687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc032440 con 0x7f19e40c7db0 2024-08-17T02:45:53.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.543+0000 7f19e1400640 1 -- 172.21.15.16:0/2642578687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0327e0 con 0x7f19e40c7db0 2024-08-17T02:45:53.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.543+0000 7f19e9000640 1 -- 172.21.15.16:0/2642578687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e40c7db0 msgr2=0x7f19e40c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:53.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.543+0000 7f19e9000640 1 --2- 172.21.15.16:0/2642578687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e40c7db0 0x7f19e40c8bd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009850 tx=0x7f19cc0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.543+0000 7f19e9000640 1 -- 172.21.15.16:0/2642578687 shutdown_connections 2024-08-17T02:45:53.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.543+0000 7f19e9000640 1 --2- 172.21.15.16:0/2642578687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e40c7db0 0x7f19e40c8bd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.543+0000 7f19e9000640 1 -- 172.21.15.16:0/2642578687 >> 172.21.15.16:0/2642578687 conn(0x7f19e4067750 msgr2=0x7f19e40c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:53.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.543+0000 7f19e9000640 1 -- 172.21.15.16:0/2642578687 shutdown_connections 2024-08-17T02:45:53.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.543+0000 7f19e9000640 1 -- 172.21.15.16:0/2642578687 wait complete. 2024-08-17T02:45:53.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.545+0000 7f19e9000640 1 Processor -- start 2024-08-17T02:45:53.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.545+0000 7f19e9000640 1 -- start start 2024-08-17T02:45:53.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.546+0000 7f19e9000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19e40c7db0 0x7f19e41562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.546+0000 7f19e9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19e4155a80 0x7f19e4157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.546+0000 7f19e9000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4155e50 0x7f19e4154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.546+0000 7f19e9000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4169190 con 0x7f19e4155e50 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.546+0000 7f19e9000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4169450 con 0x7f19e4155a80 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.546+0000 7f19e9000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4169730 con 0x7f19e40c7db0 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.546+0000 7f19e2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4155e50 0x7f19e4154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.546+0000 7f19e1e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19e40c7db0 0x7f19e41562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.546+0000 7f19dbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19e4155a80 0x7f19e4157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.546+0000 7f19e1e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19e40c7db0 0x7f19e41562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:50088/0 (socket says 172.21.15.16:50088) 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.546+0000 7f19e2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4155e50 0x7f19e4154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36668/0 (socket says 172.21.15.16:36668) 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.546+0000 7f19e1e00640 1 -- 172.21.15.16:0/2039136440 learned_addr learned my addr 172.21.15.16:0/2039136440 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.547+0000 7f19e2800640 1 -- 172.21.15.16:0/2039136440 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19e40c7db0 msgr2=0x7f19e41562e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.547+0000 7f19e2800640 1 --2- 172.21.15.16:0/2039136440 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19e40c7db0 0x7f19e41562e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.547+0000 7f19e2800640 1 -- 172.21.15.16:0/2039136440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19e4155a80 msgr2=0x7f19e4157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.547+0000 7f19e2800640 1 --2- 172.21.15.16:0/2039136440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19e4155a80 0x7f19e4157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.547+0000 7f19e2800640 1 -- 172.21.15.16:0/2039136440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc009390 con 0x7f19e4155e50 2024-08-17T02:45:53.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.547+0000 7f19e2800640 1 --2- 172.21.15.16:0/2039136440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4155e50 0x7f19e4154160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00b810 tx=0x7f19dc0092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.547+0000 7f19d9600640 1 -- 172.21.15.16:0/2039136440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00c730 con 0x7f19e4155e50 2024-08-17T02:45:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.547+0000 7f19d9600640 1 -- 172.21.15.16:0/2039136440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc00c8d0 con 0x7f19e4155e50 2024-08-17T02:45:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.547+0000 7f19d9600640 1 -- 172.21.15.16:0/2039136440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00cc90 con 0x7f19e4155e50 2024-08-17T02:45:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.547+0000 7f19e9000640 1 -- 172.21.15.16:0/2039136440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e4169a10 con 0x7f19e4155e50 2024-08-17T02:45:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.547+0000 7f19e9000640 1 -- 172.21.15.16:0/2039136440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e416a060 con 0x7f19e4155e50 2024-08-17T02:45:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.548+0000 7f19e9000640 1 -- 172.21.15.16:0/2039136440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e416c790 con 0x7f19e4155e50 2024-08-17T02:45:53.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.549+0000 7f19d9600640 1 -- 172.21.15.16:0/2039136440 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f19dc014620 con 0x7f19e4155e50 2024-08-17T02:45:53.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.550+0000 7f19d9600640 1 --2- 172.21.15.16:0/2039136440 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f19c0065150 0x7f19c0067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:53.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.550+0000 7f19d9600640 1 -- 172.21.15.16:0/2039136440 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f19dc081de0 con 0x7f19e4155e50 2024-08-17T02:45:53.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.550+0000 7f19e1e00640 1 --2- 172.21.15.16:0/2039136440 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f19c0065150 0x7f19c0067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:53.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.551+0000 7f19e1e00640 1 --2- 172.21.15.16:0/2039136440 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f19c0065150 0x7f19c0067600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f19e4152ec0 tx=0x7f19d4007d60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.554+0000 7f19d9600640 1 -- 172.21.15.16:0/2039136440 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f19dc04ce40 con 0x7f19e4155e50 2024-08-17T02:45:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.739+0000 7f19e9000640 1 -- 172.21.15.16:0/2039136440 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f19e4071050 con 0x7f19e4155e50 2024-08-17T02:45:53.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.743+0000 7f19d9600640 1 -- 172.21.15.16:0/2039136440 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f19e416c790 con 0x7f19e4155e50 2024-08-17T02:45:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.745+0000 7f19e9000640 1 -- 172.21.15.16:0/2039136440 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f19c0065150 msgr2=0x7f19c0067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.745+0000 7f19e9000640 1 --2- 172.21.15.16:0/2039136440 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f19c0065150 0x7f19c0067600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f19e4152ec0 tx=0x7f19d4007d60 comp rx=0 tx=0).stop 2024-08-17T02:45:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.745+0000 7f19e9000640 1 -- 172.21.15.16:0/2039136440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4155e50 msgr2=0x7f19e4154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.745+0000 7f19e9000640 1 --2- 172.21.15.16:0/2039136440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4155e50 0x7f19e4154160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00b810 tx=0x7f19dc0092b0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.745+0000 7f19e9000640 1 -- 172.21.15.16:0/2039136440 shutdown_connections 2024-08-17T02:45:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.745+0000 7f19e9000640 1 --2- 172.21.15.16:0/2039136440 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f19c0065150 0x7f19c0067600 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.745+0000 7f19e9000640 1 --2- 172.21.15.16:0/2039136440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19e4155e50 0x7f19e4154160 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.745+0000 7f19e9000640 1 --2- 172.21.15.16:0/2039136440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19e4155a80 0x7f19e4157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.745+0000 7f19e9000640 1 --2- 172.21.15.16:0/2039136440 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19e40c7db0 0x7f19e41562e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.745+0000 7f19e9000640 1 -- 172.21.15.16:0/2039136440 >> 172.21.15.16:0/2039136440 conn(0x7f19e4067750 msgr2=0x7f19e40aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:53.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.745+0000 7f19e9000640 1 -- 172.21.15.16:0/2039136440 shutdown_connections 2024-08-17T02:45:53.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:53.746+0000 7f19e9000640 1 -- 172.21.15.16:0/2039136440 wait complete. 2024-08-17T02:45:53.762 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-08-17T02:45:53.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-17T02:45:53.862 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-17T02:45:53.955 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-08-17T02:45:53.955 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-08-17T02:45:53.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:45:53.955 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-08-17T02:45:54.017 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:45:54.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:54.017 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-17T02:45:54.064 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-08-17T02:45:54.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:45:54.065 DEBUG:teuthology.orchestra.run.smithi142:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-08-17T02:45:54.130 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:45:54.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:54.130 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-08-17T02:45:54.184 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-08-17T02:45:54.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:45:54.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-08-17T02:45:54.244 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:45:54.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:54.245 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-08-17T02:45:54.280 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-08-17T02:45:54.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:45:54.281 DEBUG:teuthology.orchestra.run.smithi142:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-08-17T02:45:54.341 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:45:54.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:45:54.341 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-08-17T02:45:54.378 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-08-17T02:45:54.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:45:54.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.528+0000 7f0097e00640 1 Processor -- start 2024-08-17T02:45:54.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.528+0000 7f0097e00640 1 -- start start 2024-08-17T02:45:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.529+0000 7f0097e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0090155180 0x7f00900aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.529+0000 7f0097e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f009006e080 0x7f009006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.529+0000 7f0097e00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00900afc10 con 0x7f0090155550 2024-08-17T02:45:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.529+0000 7f0097e00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009006ea00 con 0x7f0090155180 2024-08-17T02:45:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.529+0000 7f0097e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00900af000 con 0x7f009006e080 2024-08-17T02:45:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.529+0000 7f0095400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0090155180 0x7f00900aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.529+0000 7f0094a00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0090155550 0x7f00900ae070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56162/0 (socket says 172.21.15.16:56162) 2024-08-17T02:45:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.529+0000 7f0094a00640 1 -- 172.21.15.16:0/3562852521 learned_addr learned my addr 172.21.15.16:0/3562852521 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.530+0000 7f0095e00640 1 --2- 172.21.15.16:0/3562852521 >> v2:172.21.15.142:3300/0 conn(0x7f009006e080 0x7f009006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.530+0000 7f0095400640 1 -- 172.21.15.16:0/3562852521 >> v2:172.21.15.142:3300/0 conn(0x7f009006e080 msgr2=0x7f009006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.530+0000 7f0095400640 1 --2- 172.21.15.16:0/3562852521 >> v2:172.21.15.142:3300/0 conn(0x7f009006e080 0x7f009006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.530+0000 7f0095400640 1 -- 172.21.15.16:0/3562852521 >> v1:172.21.15.16:6789/0 conn(0x7f0090155550 legacy=0x7f00900ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.530+0000 7f008fe00640 1 -- 172.21.15.16:0/3562852521 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888034614 0 0) 0x7f00900afc10 con 0x7f0090155550 2024-08-17T02:45:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.530+0000 7f0095400640 1 -- 172.21.15.16:0/3562852521 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00900af2e0 con 0x7f0090155180 2024-08-17T02:45:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.530+0000 7f0095e00640 1 --2- 172.21.15.16:0/3562852521 >> v2:172.21.15.142:3300/0 conn(0x7f009006e080 0x7f009006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.531+0000 7f0095400640 1 --2- 172.21.15.16:0/3562852521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0090155180 0x7f00900aea20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f00900642c0 tx=0x7f00840090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897ae0ad9e801337 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.531+0000 7f008fe00640 1 -- 172.21.15.16:0/3562852521 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008402adb0 con 0x7f0090155180 2024-08-17T02:45:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.531+0000 7f008fe00640 1 -- 172.21.15.16:0/3562852521 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084032440 con 0x7f0090155180 2024-08-17T02:45:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.531+0000 7f008fe00640 1 -- 172.21.15.16:0/3562852521 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00840327e0 con 0x7f0090155180 2024-08-17T02:45:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.532+0000 7f0097e00640 1 -- 172.21.15.16:0/3562852521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0090155180 msgr2=0x7f00900aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.532+0000 7f0097e00640 1 --2- 172.21.15.16:0/3562852521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0090155180 0x7f00900aea20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f00900642c0 tx=0x7f00840090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.532+0000 7f0097e00640 1 -- 172.21.15.16:0/3562852521 shutdown_connections 2024-08-17T02:45:54.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.532+0000 7f0097e00640 1 --2- 172.21.15.16:0/3562852521 >> v2:172.21.15.142:3300/0 conn(0x7f009006e080 0x7f009006e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.532+0000 7f0097e00640 1 --2- 172.21.15.16:0/3562852521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0090155180 0x7f00900aea20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.532+0000 7f0097e00640 1 -- 172.21.15.16:0/3562852521 >> 172.21.15.16:0/3562852521 conn(0x7f00900cb9b0 msgr2=0x7f00900cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:54.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.532+0000 7f0097e00640 1 -- 172.21.15.16:0/3562852521 shutdown_connections 2024-08-17T02:45:54.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.532+0000 7f0097e00640 1 -- 172.21.15.16:0/3562852521 wait complete. 2024-08-17T02:45:54.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.533+0000 7f0097e00640 1 Processor -- start 2024-08-17T02:45:54.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.533+0000 7f0097e00640 1 -- start start 2024-08-17T02:45:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.534+0000 7f0097e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f009006e080 0x7f00900c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.534+0000 7f0097e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0090155180 0x7f0090150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.534+0000 7f0097e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0090155550 0x7f00900b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.534+0000 7f0097e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0090167c60 con 0x7f0090155550 2024-08-17T02:45:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.534+0000 7f0097e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00900c79b0 con 0x7f0090155180 2024-08-17T02:45:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.534+0000 7f0097e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0090167050 con 0x7f009006e080 2024-08-17T02:45:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.534+0000 7f0095e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0090155550 0x7f00900b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.534+0000 7f0095e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0090155550 0x7f00900b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49274/0 (socket says 172.21.15.16:49274) 2024-08-17T02:45:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.534+0000 7f0095e00640 1 -- 172.21.15.16:0/3858545835 learned_addr learned my addr 172.21.15.16:0/3858545835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.534+0000 7f0095400640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f009006e080 0x7f00900c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.535+0000 7f0094a00640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0090155180 0x7f0090150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.535+0000 7f0095e00640 1 -- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f009006e080 msgr2=0x7f00900c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.535+0000 7f0095e00640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f009006e080 0x7f00900c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.535+0000 7f0095e00640 1 -- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0090155180 msgr2=0x7f0090150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.535+0000 7f0095e00640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0090155180 0x7f0090150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.535+0000 7f0095e00640 1 -- 172.21.15.16:0/3858545835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0084009390 con 0x7f0090155550 2024-08-17T02:45:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.535+0000 7f0095e00640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0090155550 0x7f00900b10a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f008800d1b0 tx=0x7f0088009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.535+0000 7f008cc00640 1 -- 172.21.15.16:0/3858545835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088015440 con 0x7f0090155550 2024-08-17T02:45:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.535+0000 7f008cc00640 1 -- 172.21.15.16:0/3858545835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008800f040 con 0x7f0090155550 2024-08-17T02:45:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.535+0000 7f008cc00640 1 -- 172.21.15.16:0/3858545835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088015810 con 0x7f0090155550 2024-08-17T02:45:54.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.535+0000 7f0097e00640 1 -- 172.21.15.16:0/3858545835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0090167330 con 0x7f0090155550 2024-08-17T02:45:54.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.536+0000 7f0097e00640 1 -- 172.21.15.16:0/3858545835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00901678f0 con 0x7f0090155550 2024-08-17T02:45:54.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.537+0000 7f0097e00640 1 -- 172.21.15.16:0/3858545835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0090064db0 con 0x7f0090155550 2024-08-17T02:45:54.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.537+0000 7f008cc00640 1 -- 172.21.15.16:0/3858545835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f008801e3b0 con 0x7f0090155550 2024-08-17T02:45:54.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.538+0000 7f008cc00640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0070065150 0x7f0070067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:54.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.538+0000 7f008cc00640 1 -- 172.21.15.16:0/3858545835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0088082e80 con 0x7f0090155550 2024-08-17T02:45:54.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.538+0000 7f0095400640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0070065150 0x7f0070067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:54.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.539+0000 7f0095400640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0070065150 0x7f0070067600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0084034000 tx=0x7f0084032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:54.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.542+0000 7f008cc00640 1 -- 172.21.15.16:0/3858545835 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0088050f50 con 0x7f0090155550 2024-08-17T02:45:54.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.762+0000 7f0097e00640 1 -- 172.21.15.16:0/3858545835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0090153120 con 0x7f0090155550 2024-08-17T02:45:54.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.763+0000 7f008cc00640 1 -- 172.21.15.16:0/3858545835 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f0088054570 con 0x7f0090155550 2024-08-17T02:45:54.764 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:54.764 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/111049205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":111049205},{"type":"v1","addr":"172.21.15.142:6836","nonce":111049205}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2692359383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2692359383},{"type":"v1","addr":"172.21.15.142:6837","nonce":2692359383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3179355872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3179355872},{"type":"v1","addr":"172.21.15.16:6833","nonce":3179355872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:45:19.450658+0000","modified":"2024-08-17T02:45:23.039406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2140506421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2140506421},{"type":"v1","addr":"172.21.15.16:6835","nonce":2140506421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:45:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-08-17T02:45:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.765+0000 7f0097e00640 1 -- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0070065150 msgr2=0x7f0070067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.765+0000 7f0097e00640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0070065150 0x7f0070067600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0084034000 tx=0x7f0084032000 comp rx=0 tx=0).stop 2024-08-17T02:45:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.765+0000 7f0097e00640 1 -- 172.21.15.16:0/3858545835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0090155550 msgr2=0x7f00900b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.765+0000 7f0097e00640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0090155550 0x7f00900b10a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f008800d1b0 tx=0x7f0088009a50 comp rx=0 tx=0).stop 2024-08-17T02:45:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.765+0000 7f0097e00640 1 -- 172.21.15.16:0/3858545835 shutdown_connections 2024-08-17T02:45:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.765+0000 7f0097e00640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0070065150 0x7f0070067600 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.765+0000 7f0097e00640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0090155550 0x7f00900b10a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.765+0000 7f0097e00640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0090155180 0x7f0090150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.765+0000 7f0097e00640 1 --2- 172.21.15.16:0/3858545835 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f009006e080 0x7f00900c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.765+0000 7f0097e00640 1 -- 172.21.15.16:0/3858545835 >> 172.21.15.16:0/3858545835 conn(0x7f00900cb9b0 msgr2=0x7f0090154210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.765+0000 7f0097e00640 1 -- 172.21.15.16:0/3858545835 shutdown_connections 2024-08-17T02:45:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.765+0000 7f0097e00640 1 -- 172.21.15.16:0/3858545835 wait complete. 2024-08-17T02:45:54.782 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:54.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.933+0000 7f1021400640 1 Processor -- start 2024-08-17T02:45:54.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.933+0000 7f1021400640 1 -- start start 2024-08-17T02:45:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.934+0000 7f1021400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f101c06df00 0x7f101c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.934+0000 7f1021400640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c156670 con 0x7f101c155a20 2024-08-17T02:45:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.934+0000 7f1021400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101c06e5e0 con 0x7f101c155df0 2024-08-17T02:45:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.934+0000 7f1021400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f101c0b0350 con 0x7f101c06df00 2024-08-17T02:45:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.935+0000 7f101a800640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f101c155a20 0x7f101c06d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56174/0 (socket says 172.21.15.16:56174) 2024-08-17T02:45:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.935+0000 7f101a800640 1 -- 172.21.15.16:0/1012317762 learned_addr learned my addr 172.21.15.16:0/1012317762 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:54.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.935+0000 7f1019e00640 1 --2- 172.21.15.16:0/1012317762 >> v2:172.21.15.16:3300/0 conn(0x7f101c06df00 0x7f101c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.935+0000 7f1019400640 1 -- 172.21.15.16:0/1012317762 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233478700 0 0) 0x7f101c156670 con 0x7f101c155a20 2024-08-17T02:45:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.935+0000 7f1019400640 1 -- 172.21.15.16:0/1012317762 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10000034e0 con 0x7f101c155a20 2024-08-17T02:45:54.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.935+0000 7f1019e00640 1 -- 172.21.15.16:0/1012317762 >> v1:172.21.15.142:6789/0 conn(0x7f101c155df0 legacy=0x7f101c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:54.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.935+0000 7f1019e00640 1 -- 172.21.15.16:0/1012317762 >> v1:172.21.15.16:6789/0 conn(0x7f101c155a20 legacy=0x7f101c06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:54.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.935+0000 7f1019e00640 1 -- 172.21.15.16:0/1012317762 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c06e8c0 con 0x7f101c06df00 2024-08-17T02:45:54.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.936+0000 7f1019e00640 1 --2- 172.21.15.16:0/1012317762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c06df00 0x7f101c0adf10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1008009850 tx=0x7f10080090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b289a69a0c244bf server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:54.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.936+0000 7f1019400640 1 -- 172.21.15.16:0/1012317762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100802ad90 con 0x7f101c06df00 2024-08-17T02:45:54.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.937+0000 7f1019400640 1 -- 172.21.15.16:0/1012317762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1008032440 con 0x7f101c06df00 2024-08-17T02:45:54.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.937+0000 7f1019400640 1 -- 172.21.15.16:0/1012317762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080327e0 con 0x7f101c06df00 2024-08-17T02:45:54.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.937+0000 7f1021400640 1 -- 172.21.15.16:0/1012317762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c06df00 msgr2=0x7f101c0adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:54.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.937+0000 7f1021400640 1 --2- 172.21.15.16:0/1012317762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c06df00 0x7f101c0adf10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1008009850 tx=0x7f10080090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.937+0000 7f1021400640 1 -- 172.21.15.16:0/1012317762 shutdown_connections 2024-08-17T02:45:54.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.937+0000 7f1021400640 1 --2- 172.21.15.16:0/1012317762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c06df00 0x7f101c0adf10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.937+0000 7f1021400640 1 -- 172.21.15.16:0/1012317762 >> 172.21.15.16:0/1012317762 conn(0x7f101c0cb9b0 msgr2=0x7f101c0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:54.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.937+0000 7f1021400640 1 -- 172.21.15.16:0/1012317762 shutdown_connections 2024-08-17T02:45:54.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.937+0000 7f1021400640 1 -- 172.21.15.16:0/1012317762 wait complete. 2024-08-17T02:45:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.939+0000 7f1021400640 1 Processor -- start 2024-08-17T02:45:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.939+0000 7f1021400640 1 -- start start 2024-08-17T02:45:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.940+0000 7f1021400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f101c06df00 0x7f101c14f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.940+0000 7f1021400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f101c155a20 0x7f101c169200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.940+0000 7f1021400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c155df0 0x7f101c0a7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.940+0000 7f1021400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c14fb10 con 0x7f101c155df0 2024-08-17T02:45:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.940+0000 7f1021400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101c0ad2b0 con 0x7f101c155a20 2024-08-17T02:45:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.940+0000 7f1021400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101c169740 con 0x7f101c06df00 2024-08-17T02:45:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.940+0000 7f101b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c155df0 0x7f101c0a7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.940+0000 7f101b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c155df0 0x7f101c0a7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49292/0 (socket says 172.21.15.16:49292) 2024-08-17T02:45:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.940+0000 7f101b200640 1 -- 172.21.15.16:0/2547201302 learned_addr learned my addr 172.21.15.16:0/2547201302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:54.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.940+0000 7f1019e00640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f101c155a20 0x7f101c169200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.940+0000 7f101a800640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f101c06df00 0x7f101c14f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.940+0000 7f101b200640 1 -- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f101c06df00 msgr2=0x7f101c14f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.941+0000 7f101b200640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f101c06df00 0x7f101c14f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.941+0000 7f101b200640 1 -- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f101c155a20 msgr2=0x7f101c169200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.941+0000 7f101b200640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f101c155a20 0x7f101c169200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.941+0000 7f101b200640 1 -- 172.21.15.16:0/2547201302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1008009390 con 0x7f101c155df0 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.941+0000 7f101a800640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f101c06df00 0x7f101c14f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.941+0000 7f1019e00640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f101c155a20 0x7f101c169200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.941+0000 7f101b200640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c155df0 0x7f101c0a7bc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f101400c7f0 tx=0x7f10140015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.941+0000 7f1012000640 1 -- 172.21.15.16:0/2547201302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400a640 con 0x7f101c155df0 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.941+0000 7f1012000640 1 -- 172.21.15.16:0/2547201302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101400a7c0 con 0x7f101c155df0 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.941+0000 7f1012000640 1 -- 172.21.15.16:0/2547201302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400ab80 con 0x7f101c155df0 2024-08-17T02:45:54.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.941+0000 7f1021400640 1 -- 172.21.15.16:0/2547201302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101c0ac6e0 con 0x7f101c155df0 2024-08-17T02:45:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.941+0000 7f1021400640 1 -- 172.21.15.16:0/2547201302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101c0acc60 con 0x7f101c155df0 2024-08-17T02:45:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.943+0000 7f1021400640 1 -- 172.21.15.16:0/2547201302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101c064db0 con 0x7f101c155df0 2024-08-17T02:45:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.943+0000 7f1012000640 1 -- 172.21.15.16:0/2547201302 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1014012630 con 0x7f101c155df0 2024-08-17T02:45:54.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.944+0000 7f1012000640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0ffc065150 0x7f0ffc067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:54.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.944+0000 7f1012000640 1 -- 172.21.15.16:0/2547201302 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1014082800 con 0x7f101c155df0 2024-08-17T02:45:54.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.944+0000 7f101a800640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0ffc065150 0x7f0ffc067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:54.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.945+0000 7f101a800640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0ffc065150 0x7f0ffc067600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f100c007fd0 tx=0x7f100c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:54.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:54.948+0000 7f1012000640 1 -- 172.21.15.16:0/2547201302 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f10140508c0 con 0x7f101c155df0 2024-08-17T02:45:55.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.132+0000 7f1021400640 1 -- 172.21.15.16:0/2547201302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f101c068850 con 0x7f101c155df0 2024-08-17T02:45:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.133+0000 7f1012000640 1 -- 172.21.15.16:0/2547201302 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f1014053ee0 con 0x7f101c155df0 2024-08-17T02:45:55.134 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:55.135 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.135+0000 7f1021400640 1 -- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0ffc065150 msgr2=0x7f0ffc067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.135+0000 7f1021400640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0ffc065150 0x7f0ffc067600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f100c007fd0 tx=0x7f100c009040 comp rx=0 tx=0).stop 2024-08-17T02:45:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.135+0000 7f1021400640 1 -- 172.21.15.16:0/2547201302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c155df0 msgr2=0x7f101c0a7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.135+0000 7f1021400640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c155df0 0x7f101c0a7bc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f101400c7f0 tx=0x7f10140015b0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.135+0000 7f1021400640 1 -- 172.21.15.16:0/2547201302 shutdown_connections 2024-08-17T02:45:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.135+0000 7f1021400640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0ffc065150 0x7f0ffc067600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.135+0000 7f1021400640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c155df0 0x7f101c0a7bc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.135+0000 7f1021400640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f101c155a20 0x7f101c169200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.135+0000 7f1021400640 1 --2- 172.21.15.16:0/2547201302 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f101c06df00 0x7f101c14f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.135+0000 7f1021400640 1 -- 172.21.15.16:0/2547201302 >> 172.21.15.16:0/2547201302 conn(0x7f101c0cb9b0 msgr2=0x7f101c156e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.135+0000 7f1021400640 1 -- 172.21.15.16:0/2547201302 shutdown_connections 2024-08-17T02:45:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.136+0000 7f1021400640 1 -- 172.21.15.16:0/2547201302 wait complete. 2024-08-17T02:45:55.152 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:45:14.291903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:45:18.004797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:45:18.430718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-17T02:45:55.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:45:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.308+0000 7f819f200640 1 Processor -- start 2024-08-17T02:45:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.308+0000 7f819f200640 1 -- start start 2024-08-17T02:45:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.310+0000 7f819f200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81980b0a80 0x7f81980b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.310+0000 7f819f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8198157e10 0x7f81980ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.310+0000 7f819f200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81980ae150 con 0x7f81980adcf0 2024-08-17T02:45:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.310+0000 7f819f200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81980b1390 con 0x7f81980b0a80 2024-08-17T02:45:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.310+0000 7f819f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8198157070 con 0x7f8198157e10 2024-08-17T02:45:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.310+0000 7f8197e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81980b0a80 0x7f81980b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.310+0000 7f8197e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81980b0a80 0x7f81980b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49308/0 (socket says 172.21.15.16:49308) 2024-08-17T02:45:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.310+0000 7f8197e00640 1 -- 172.21.15.16:0/2195166703 learned_addr learned my addr 172.21.15.16:0/2195166703 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.310+0000 7f818fe00640 1 --2- 172.21.15.16:0/2195166703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8198157e10 0x7f81980ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.311+0000 7f8197e00640 1 -- 172.21.15.16:0/2195166703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8198157e10 msgr2=0x7f81980ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.311+0000 7f8197e00640 1 --2- 172.21.15.16:0/2195166703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8198157e10 0x7f81980ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.311+0000 7f8197e00640 1 -- 172.21.15.16:0/2195166703 >> v1:172.21.15.142:6789/0 conn(0x7f81980adcf0 legacy=0x7f819815a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.311+0000 7f8197e00640 1 -- 172.21.15.16:0/2195166703 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819815b410 con 0x7f81980b0a80 2024-08-17T02:45:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.312+0000 7f8197e00640 1 --2- 172.21.15.16:0/2195166703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81980b0a80 0x7f81980b0e50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8188009910 tx=0x7f81880091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95196355175973f server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:55.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.312+0000 7f8197400640 1 -- 172.21.15.16:0/2195166703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818802add0 con 0x7f81980b0a80 2024-08-17T02:45:55.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.312+0000 7f8197400640 1 -- 172.21.15.16:0/2195166703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8188032440 con 0x7f81980b0a80 2024-08-17T02:45:55.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.312+0000 7f8197400640 1 -- 172.21.15.16:0/2195166703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188032820 con 0x7f81980b0a80 2024-08-17T02:45:55.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.313+0000 7f819f200640 1 -- 172.21.15.16:0/2195166703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81980b0a80 msgr2=0x7f81980b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.313+0000 7f819f200640 1 --2- 172.21.15.16:0/2195166703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81980b0a80 0x7f81980b0e50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8188009910 tx=0x7f81880091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.313+0000 7f819f200640 1 -- 172.21.15.16:0/2195166703 shutdown_connections 2024-08-17T02:45:55.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.313+0000 7f819f200640 1 --2- 172.21.15.16:0/2195166703 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8198157e10 0x7f81980ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.313+0000 7f819f200640 1 --2- 172.21.15.16:0/2195166703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81980b0a80 0x7f81980b0e50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.313+0000 7f819f200640 1 -- 172.21.15.16:0/2195166703 >> 172.21.15.16:0/2195166703 conn(0x7f8198067750 msgr2=0x7f81980cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.313+0000 7f819f200640 1 -- 172.21.15.16:0/2195166703 shutdown_connections 2024-08-17T02:45:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.313+0000 7f819f200640 1 -- 172.21.15.16:0/2195166703 wait complete. 2024-08-17T02:45:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.314+0000 7f819f200640 1 Processor -- start 2024-08-17T02:45:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.314+0000 7f819f200640 1 -- start start 2024-08-17T02:45:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.315+0000 7f819f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81980adcf0 0x7f81981505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.315+0000 7f819f200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81980b0a80 0x7f819814c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.315+0000 7f819f200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8198157e10 0x7f81980c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.315+0000 7f819f200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81980c1a90 con 0x7f81980b0a80 2024-08-17T02:45:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.315+0000 7f819f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81980c0e80 con 0x7f81980adcf0 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.315+0000 7f819f200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81980c1160 con 0x7f8198157e10 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.315+0000 7f818fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81980b0a80 0x7f819814c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.316+0000 7f818fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81980b0a80 0x7f819814c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49314/0 (socket says 172.21.15.16:49314) 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.316+0000 7f818fe00640 1 -- 172.21.15.16:0/3442172681 learned_addr learned my addr 172.21.15.16:0/3442172681 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.316+0000 7f8197e00640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81980adcf0 0x7f81981505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.316+0000 7f819d200640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8198157e10 0x7f81980c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.316+0000 7f818fe00640 1 -- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8198157e10 msgr2=0x7f81980c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.316+0000 7f818fe00640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8198157e10 0x7f81980c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.316+0000 7f818fe00640 1 -- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81980adcf0 msgr2=0x7f81981505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.316+0000 7f818fe00640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81980adcf0 0x7f81981505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.316+0000 7f818fe00640 1 -- 172.21.15.16:0/3442172681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8188009450 con 0x7f81980b0a80 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.316+0000 7f8197e00640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81980adcf0 0x7f81981505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.316+0000 7f818fe00640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81980b0a80 0x7f819814c610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f818000d890 tx=0x7f8180008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.317+0000 7f818f400640 1 -- 172.21.15.16:0/3442172681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000de20 con 0x7f81980b0a80 2024-08-17T02:45:55.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.317+0000 7f818f400640 1 -- 172.21.15.16:0/3442172681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818000b8d0 con 0x7f81980b0a80 2024-08-17T02:45:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.317+0000 7f818f400640 1 -- 172.21.15.16:0/3442172681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818000bc90 con 0x7f81980b0a80 2024-08-17T02:45:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.317+0000 7f819f200640 1 -- 172.21.15.16:0/3442172681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81980c1440 con 0x7f81980b0a80 2024-08-17T02:45:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.317+0000 7f819f200640 1 -- 172.21.15.16:0/3442172681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8198150ae0 con 0x7f81980b0a80 2024-08-17T02:45:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.318+0000 7f819f200640 1 -- 172.21.15.16:0/3442172681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81980c1c00 con 0x7f81980b0a80 2024-08-17T02:45:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.319+0000 7f818f400640 1 -- 172.21.15.16:0/3442172681 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f818001e460 con 0x7f81980b0a80 2024-08-17T02:45:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.319+0000 7f818f400640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8174065150 0x7f8174067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.319+0000 7f818f400640 1 -- 172.21.15.16:0/3442172681 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8180082a30 con 0x7f81980b0a80 2024-08-17T02:45:55.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.320+0000 7f8197e00640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8174065150 0x7f8174067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:55.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.321+0000 7f8197e00640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8174065150 0x7f8174067600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8188034000 tx=0x7f8188032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.323+0000 7f818f400640 1 -- 172.21.15.16:0/3442172681 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8180050b00 con 0x7f81980b0a80 2024-08-17T02:45:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.508+0000 7f819f200640 1 -- 172.21.15.16:0/3442172681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8198064db0 con 0x7f81980b0a80 2024-08-17T02:45:55.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.509+0000 7f818f400640 1 -- 172.21.15.16:0/3442172681 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8180054120 con 0x7f81980b0a80 2024-08-17T02:45:55.510 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-17T02:45:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.511+0000 7f819f200640 1 -- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8174065150 msgr2=0x7f8174067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.511+0000 7f819f200640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8174065150 0x7f8174067600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8188034000 tx=0x7f8188032000 comp rx=0 tx=0).stop 2024-08-17T02:45:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.511+0000 7f819f200640 1 -- 172.21.15.16:0/3442172681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81980b0a80 msgr2=0x7f819814c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.511+0000 7f819f200640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81980b0a80 0x7f819814c610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f818000d890 tx=0x7f8180008040 comp rx=0 tx=0).stop 2024-08-17T02:45:55.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.511+0000 7f819f200640 1 -- 172.21.15.16:0/3442172681 shutdown_connections 2024-08-17T02:45:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.511+0000 7f819f200640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8174065150 0x7f8174067600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.511+0000 7f819f200640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8198157e10 0x7f81980c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.511+0000 7f819f200640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81980b0a80 0x7f819814c610 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.511+0000 7f819f200640 1 --2- 172.21.15.16:0/3442172681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81980adcf0 0x7f81981505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.511+0000 7f819f200640 1 -- 172.21.15.16:0/3442172681 >> 172.21.15.16:0/3442172681 conn(0x7f8198067750 msgr2=0x7f819814d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.511+0000 7f819f200640 1 -- 172.21.15.16:0/3442172681 shutdown_connections 2024-08-17T02:45:55.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.511+0000 7f819f200640 1 -- 172.21.15.16:0/3442172681 wait complete. 2024-08-17T02:45:55.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:45:55.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.706+0000 7f4985200640 1 Processor -- start 2024-08-17T02:45:55.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.706+0000 7f4985200640 1 -- start start 2024-08-17T02:45:55.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.707+0000 7f4985200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4980158a30 0x7f49800c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:55.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.707+0000 7f4985200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49800c61d0 con 0x7f49800c4a20 2024-08-17T02:45:55.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.707+0000 7f4985200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49800c64b0 con 0x7f49800ce890 2024-08-17T02:45:55.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.707+0000 7f4985200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49800c6790 con 0x7f4980158a30 2024-08-17T02:45:55.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.707+0000 7f497f400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4980158a30 0x7f49800c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:55.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.708+0000 7f497ea00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f49800c4a20 0x7f49800ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56184/0 (socket says 172.21.15.16:56184) 2024-08-17T02:45:55.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.708+0000 7f497ea00640 1 -- 172.21.15.16:0/145049194 learned_addr learned my addr 172.21.15.16:0/145049194 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:55.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.708+0000 7f497f400640 1 --2- 172.21.15.16:0/145049194 >> v2:172.21.15.16:3300/0 conn(0x7f4980158a30 0x7f49800c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49322/0 (socket says 172.21.15.16:49322) 2024-08-17T02:45:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.708+0000 7f497f400640 1 -- 172.21.15.16:0/145049194 >> v1:172.21.15.142:6789/0 conn(0x7f49800ce890 legacy=0x7f49800cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:55.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.708+0000 7f497f400640 1 -- 172.21.15.16:0/145049194 >> v1:172.21.15.16:6789/0 conn(0x7f49800c4a20 legacy=0x7f49800ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.708+0000 7f497f400640 1 -- 172.21.15.16:0/145049194 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49800cab70 con 0x7f4980158a30 2024-08-17T02:45:55.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.709+0000 7f497f400640 1 --2- 172.21.15.16:0/145049194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980158a30 0x7f49800c58d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4970009850 tx=0x7f49700090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f85542a56c80f server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:55.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.709+0000 7f497e000640 1 -- 172.21.15.16:0/145049194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497002ad00 con 0x7f4980158a30 2024-08-17T02:45:55.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.709+0000 7f497e000640 1 -- 172.21.15.16:0/145049194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497002ae60 con 0x7f4980158a30 2024-08-17T02:45:55.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.709+0000 7f497e000640 1 -- 172.21.15.16:0/145049194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4970032670 con 0x7f4980158a30 2024-08-17T02:45:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.710+0000 7f4985200640 1 -- 172.21.15.16:0/145049194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980158a30 msgr2=0x7f49800c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.710+0000 7f4985200640 1 --2- 172.21.15.16:0/145049194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980158a30 0x7f49800c58d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4970009850 tx=0x7f49700090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.710+0000 7f4985200640 1 -- 172.21.15.16:0/145049194 shutdown_connections 2024-08-17T02:45:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.710+0000 7f4985200640 1 --2- 172.21.15.16:0/145049194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980158a30 0x7f49800c58d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.710+0000 7f4985200640 1 -- 172.21.15.16:0/145049194 >> 172.21.15.16:0/145049194 conn(0x7f4980098e30 msgr2=0x7f498006eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.710+0000 7f4985200640 1 -- 172.21.15.16:0/145049194 shutdown_connections 2024-08-17T02:45:55.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.710+0000 7f4985200640 1 -- 172.21.15.16:0/145049194 wait complete. 2024-08-17T02:45:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.712+0000 7f4985200640 1 Processor -- start 2024-08-17T02:45:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.712+0000 7f4985200640 1 -- start start 2024-08-17T02:45:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.713+0000 7f4985200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49800c4a20 0x7f498016a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.713+0000 7f4985200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49800ce890 0x7f498016be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.713+0000 7f4985200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980158a30 0x7f49801570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.713+0000 7f4985200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800c8cd0 con 0x7f4980158a30 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.713+0000 7f4985200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800c83a0 con 0x7f49800ce890 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.713+0000 7f4985200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49800c8680 con 0x7f49800c4a20 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.713+0000 7f497fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980158a30 0x7f49801570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.714+0000 7f497fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980158a30 0x7f49801570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49334/0 (socket says 172.21.15.16:49334) 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.714+0000 7f497fe00640 1 -- 172.21.15.16:0/2600376276 learned_addr learned my addr 172.21.15.16:0/2600376276 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.714+0000 7f497ea00640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49800ce890 0x7f498016be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.714+0000 7f497f400640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49800c4a20 0x7f498016a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.714+0000 7f497fe00640 1 -- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49800c4a20 msgr2=0x7f498016a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.714+0000 7f497fe00640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49800c4a20 0x7f498016a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.714+0000 7f497fe00640 1 -- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49800ce890 msgr2=0x7f498016be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.714+0000 7f497fe00640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49800ce890 0x7f498016be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.714+0000 7f497fe00640 1 -- 172.21.15.16:0/2600376276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970009390 con 0x7f4980158a30 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.714+0000 7f497fe00640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980158a30 0x7f49801570c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f497800c810 tx=0x7f4978002620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.715+0000 7f4976a00640 1 -- 172.21.15.16:0/2600376276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497800ce00 con 0x7f4980158a30 2024-08-17T02:45:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.715+0000 7f4976a00640 1 -- 172.21.15.16:0/2600376276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497800a850 con 0x7f4980158a30 2024-08-17T02:45:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.715+0000 7f4976a00640 1 -- 172.21.15.16:0/2600376276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497800ac10 con 0x7f4980158a30 2024-08-17T02:45:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.715+0000 7f4985200640 1 -- 172.21.15.16:0/2600376276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49800c8960 con 0x7f4980158a30 2024-08-17T02:45:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.715+0000 7f4985200640 1 -- 172.21.15.16:0/2600376276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498016c360 con 0x7f4980158a30 2024-08-17T02:45:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.716+0000 7f4985200640 1 -- 172.21.15.16:0/2600376276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49800c8e80 con 0x7f4980158a30 2024-08-17T02:45:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.717+0000 7f4976a00640 1 -- 172.21.15.16:0/2600376276 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f4978012630 con 0x7f4980158a30 2024-08-17T02:45:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.717+0000 7f4976a00640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4960065150 0x7f4960067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.717+0000 7f4976a00640 1 -- 172.21.15.16:0/2600376276 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f49780826e0 con 0x7f4980158a30 2024-08-17T02:45:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.718+0000 7f497f400640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4960065150 0x7f4960067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.719+0000 7f497f400640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4960065150 0x7f4960067600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4970034000 tx=0x7f4970032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.721+0000 7f4976a00640 1 -- 172.21.15.16:0/2600376276 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f49800c8e80 con 0x7f4980158a30 2024-08-17T02:45:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.929+0000 7f4985200640 1 -- 172.21.15.16:0/2600376276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4980071050 con 0x7f4980158a30 2024-08-17T02:45:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.930+0000 7f4976a00640 1 -- 172.21.15.16:0/2600376276 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4978054240 con 0x7f4980158a30 2024-08-17T02:45:55.931 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:45:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.932+0000 7f4985200640 1 -- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4960065150 msgr2=0x7f4960067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.932+0000 7f4985200640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4960065150 0x7f4960067600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4970034000 tx=0x7f4970032000 comp rx=0 tx=0).stop 2024-08-17T02:45:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.932+0000 7f4985200640 1 -- 172.21.15.16:0/2600376276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980158a30 msgr2=0x7f49801570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.932+0000 7f4985200640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980158a30 0x7f49801570c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f497800c810 tx=0x7f4978002620 comp rx=0 tx=0).stop 2024-08-17T02:45:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.932+0000 7f4985200640 1 -- 172.21.15.16:0/2600376276 shutdown_connections 2024-08-17T02:45:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.932+0000 7f4985200640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4960065150 0x7f4960067600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.932+0000 7f4985200640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980158a30 0x7f49801570c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.932+0000 7f4985200640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49800ce890 0x7f498016be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.932+0000 7f4985200640 1 --2- 172.21.15.16:0/2600376276 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49800c4a20 0x7f498016a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.932+0000 7f4985200640 1 -- 172.21.15.16:0/2600376276 >> 172.21.15.16:0/2600376276 conn(0x7f4980098e30 msgr2=0x7f4980067550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.932+0000 7f4985200640 1 -- 172.21.15.16:0/2600376276 shutdown_connections 2024-08-17T02:45:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:55.932+0000 7f4985200640 1 -- 172.21.15.16:0/2600376276 wait complete. 2024-08-17T02:45:55.948 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:45:56.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.103+0000 7f5cd0200640 1 Processor -- start 2024-08-17T02:45:56.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.103+0000 7f5cd0200640 1 -- start start 2024-08-17T02:45:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.104+0000 7f5cd0200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5cc8159170 0x7f5cc80ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.104+0000 7f5cd0200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5cc80aaf20 0x7f5cc80ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.104+0000 7f5cd0200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc80681f0 con 0x7f5cc8158da0 2024-08-17T02:45:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.104+0000 7f5cd0200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc80b08c0 con 0x7f5cc8159170 2024-08-17T02:45:56.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.104+0000 7f5cd0200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc80aca00 con 0x7f5cc80aaf20 2024-08-17T02:45:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.105+0000 7f5ccce00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5cc8159170 0x7f5cc80ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.105+0000 7f5ccd800640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5cc8158da0 0x7f5cc80aa810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56196/0 (socket says 172.21.15.16:56196) 2024-08-17T02:45:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.105+0000 7f5ccd800640 1 -- 172.21.15.16:0/764816153 learned_addr learned my addr 172.21.15.16:0/764816153 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.105+0000 7f5ccce00640 1 -- 172.21.15.16:0/764816153 >> v2:172.21.15.142:3300/0 conn(0x7f5cc80aaf20 msgr2=0x7f5cc80ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.105+0000 7f5cce200640 1 --2- 172.21.15.16:0/764816153 >> v2:172.21.15.142:3300/0 conn(0x7f5cc80aaf20 0x7f5cc80ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.105+0000 7f5ccce00640 1 --2- 172.21.15.16:0/764816153 >> v2:172.21.15.142:3300/0 conn(0x7f5cc80aaf20 0x7f5cc80ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.105+0000 7f5ccce00640 1 -- 172.21.15.16:0/764816153 >> v1:172.21.15.16:6789/0 conn(0x7f5cc8158da0 legacy=0x7f5cc80aa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.105+0000 7f5ccce00640 1 -- 172.21.15.16:0/764816153 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc80acce0 con 0x7f5cc8159170 2024-08-17T02:45:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.105+0000 7f5cce200640 1 --2- 172.21.15.16:0/764816153 >> v2:172.21.15.142:3300/0 conn(0x7f5cc80aaf20 0x7f5cc80ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:56.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.105+0000 7f5cc7e00640 1 -- 172.21.15.16:0/764816153 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675043051 0 0) 0x7f5cc80681f0 con 0x7f5cc8158da0 2024-08-17T02:45:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.106+0000 7f5ccce00640 1 --2- 172.21.15.16:0/764816153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cc8159170 0x7f5cc80ac3e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80b3ba0 tx=0x7f5cb40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df3266834f1047e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.106+0000 7f5cc7e00640 1 -- 172.21.15.16:0/764816153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb402ad20 con 0x7f5cc8159170 2024-08-17T02:45:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.106+0000 7f5cc7e00640 1 -- 172.21.15.16:0/764816153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb402ae80 con 0x7f5cc8159170 2024-08-17T02:45:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.106+0000 7f5cc7e00640 1 -- 172.21.15.16:0/764816153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4032670 con 0x7f5cc8159170 2024-08-17T02:45:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.107+0000 7f5cd0200640 1 -- 172.21.15.16:0/764816153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cc8159170 msgr2=0x7f5cc80ac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.107+0000 7f5cd0200640 1 --2- 172.21.15.16:0/764816153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cc8159170 0x7f5cc80ac3e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80b3ba0 tx=0x7f5cb40090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.107+0000 7f5cd0200640 1 -- 172.21.15.16:0/764816153 shutdown_connections 2024-08-17T02:45:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.107+0000 7f5cd0200640 1 --2- 172.21.15.16:0/764816153 >> v2:172.21.15.142:3300/0 conn(0x7f5cc80aaf20 0x7f5cc80ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.107+0000 7f5cd0200640 1 --2- 172.21.15.16:0/764816153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cc8159170 0x7f5cc80ac3e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.107+0000 7f5cd0200640 1 -- 172.21.15.16:0/764816153 >> 172.21.15.16:0/764816153 conn(0x7f5cc8096a70 msgr2=0x7f5cc8096e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.107+0000 7f5cd0200640 1 -- 172.21.15.16:0/764816153 shutdown_connections 2024-08-17T02:45:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.107+0000 7f5cd0200640 1 -- 172.21.15.16:0/764816153 wait complete. 2024-08-17T02:45:56.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.109+0000 7f5cd0200640 1 Processor -- start 2024-08-17T02:45:56.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.109+0000 7f5cd0200640 1 -- start start 2024-08-17T02:45:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5cd0200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cc80aaf20 0x7f5cc81684b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5cd0200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc8158da0 0x7f5cc8143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5cd0200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8145050 0x7f5cc8143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5cd0200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8143fb0 con 0x7f5cc80aaf20 2024-08-17T02:45:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5cd0200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8144290 con 0x7f5cc8145050 2024-08-17T02:45:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5cd0200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc8144570 con 0x7f5cc8158da0 2024-08-17T02:45:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5ccd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cc80aaf20 0x7f5cc81684b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5ccd800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cc80aaf20 0x7f5cc81684b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49348/0 (socket says 172.21.15.16:49348) 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5ccd800640 1 -- 172.21.15.16:0/2569078034 learned_addr learned my addr 172.21.15.16:0/2569078034 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5ccce00640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc8158da0 0x7f5cc8143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5cce200640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8145050 0x7f5cc8143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5ccd800640 1 -- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc8158da0 msgr2=0x7f5cc8143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5ccd800640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc8158da0 0x7f5cc8143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5ccd800640 1 -- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8145050 msgr2=0x7f5cc8143780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5ccd800640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8145050 0x7f5cc8143780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.110+0000 7f5ccd800640 1 -- 172.21.15.16:0/2569078034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb4009390 con 0x7f5cc80aaf20 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.111+0000 7f5ccd800640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cc80aaf20 0x7f5cc81684b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00c5e0 tx=0x7f5cbc007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.111+0000 7f5cc4c00640 1 -- 172.21.15.16:0/2569078034 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc005aa0 con 0x7f5cc80aaf20 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.111+0000 7f5cc4c00640 1 -- 172.21.15.16:0/2569078034 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc005c00 con 0x7f5cc80aaf20 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.111+0000 7f5cc4c00640 1 -- 172.21.15.16:0/2569078034 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0070f0 con 0x7f5cc80aaf20 2024-08-17T02:45:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.111+0000 7f5cd0200640 1 -- 172.21.15.16:0/2569078034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc8144850 con 0x7f5cc80aaf20 2024-08-17T02:45:56.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.111+0000 7f5cd0200640 1 -- 172.21.15.16:0/2569078034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc8143cc0 con 0x7f5cc80aaf20 2024-08-17T02:45:56.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.112+0000 7f5cd0200640 1 -- 172.21.15.16:0/2569078034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc8144d50 con 0x7f5cc80aaf20 2024-08-17T02:45:56.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.113+0000 7f5cc4c00640 1 -- 172.21.15.16:0/2569078034 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f5cbc007520 con 0x7f5cc80aaf20 2024-08-17T02:45:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.114+0000 7f5cc4c00640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ca8065180 0x7f5ca8067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.114+0000 7f5cc4c00640 1 -- 172.21.15.16:0/2569078034 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5cbc081f50 con 0x7f5cc80aaf20 2024-08-17T02:45:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.114+0000 7f5ccce00640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ca8065180 0x7f5ca8067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.115+0000 7f5ccce00640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ca8065180 0x7f5ca8067630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4034000 tx=0x7f5cb4032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.118+0000 7f5cc4c00640 1 -- 172.21.15.16:0/2569078034 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5cbc04fff0 con 0x7f5cc80aaf20 2024-08-17T02:45:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.299+0000 7f5cd0200640 1 -- 172.21.15.16:0/2569078034 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5cc8071050 con 0x7f5cc80aaf20 2024-08-17T02:45:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.299+0000 7f5cc4c00640 1 -- 172.21.15.16:0/2569078034 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5cbc053610 con 0x7f5cc80aaf20 2024-08-17T02:45:56.300 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-08-17T02:45:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.301+0000 7f5cd0200640 1 -- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ca8065180 msgr2=0x7f5ca8067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.301+0000 7f5cd0200640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ca8065180 0x7f5ca8067630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4034000 tx=0x7f5cb4032000 comp rx=0 tx=0).stop 2024-08-17T02:45:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.301+0000 7f5cd0200640 1 -- 172.21.15.16:0/2569078034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cc80aaf20 msgr2=0x7f5cc81684b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.301+0000 7f5cd0200640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cc80aaf20 0x7f5cc81684b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00c5e0 tx=0x7f5cbc007d60 comp rx=0 tx=0).stop 2024-08-17T02:45:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.301+0000 7f5cd0200640 1 -- 172.21.15.16:0/2569078034 shutdown_connections 2024-08-17T02:45:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.301+0000 7f5cd0200640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5ca8065180 0x7f5ca8067630 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.301+0000 7f5cd0200640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cc8145050 0x7f5cc8143780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.301+0000 7f5cd0200640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5cc8158da0 0x7f5cc8143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.301+0000 7f5cd0200640 1 --2- 172.21.15.16:0/2569078034 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cc80aaf20 0x7f5cc81684b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.301+0000 7f5cd0200640 1 -- 172.21.15.16:0/2569078034 >> 172.21.15.16:0/2569078034 conn(0x7f5cc8096a70 msgr2=0x7f5cc8156ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.301+0000 7f5cd0200640 1 -- 172.21.15.16:0/2569078034 shutdown_connections 2024-08-17T02:45:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.301+0000 7f5cd0200640 1 -- 172.21.15.16:0/2569078034 wait complete. 2024-08-17T02:45:56.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:45:56.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.468+0000 7f346a800640 1 Processor -- start 2024-08-17T02:45:56.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.468+0000 7f346a800640 1 -- start start 2024-08-17T02:45:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.469+0000 7f346a800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3464159ca0 0x7f34640ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.469+0000 7f346a800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f34640ca8a0 0x7f34640cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.469+0000 7f346a800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34640ccf30 con 0x7f34640ccad0 2024-08-17T02:45:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.470+0000 7f346a800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34640cd6e0 con 0x7f3464159ca0 2024-08-17T02:45:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.470+0000 7f346a800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34640cadc0 con 0x7f34640ca8a0 2024-08-17T02:45:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.470+0000 7f3469400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3464159ca0 0x7f34640ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.470+0000 7f3469400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3464159ca0 0x7f34640ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49354/0 (socket says 172.21.15.16:49354) 2024-08-17T02:45:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.470+0000 7f3469400640 1 -- 172.21.15.16:0/2093738925 learned_addr learned my addr 172.21.15.16:0/2093738925 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.470+0000 7f3468a00640 1 --2- 172.21.15.16:0/2093738925 >> v2:172.21.15.142:3300/0 conn(0x7f34640ca8a0 0x7f34640cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.470+0000 7f3469400640 1 -- 172.21.15.16:0/2093738925 >> v2:172.21.15.142:3300/0 conn(0x7f34640ca8a0 msgr2=0x7f34640cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:56.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.470+0000 7f3469400640 1 --2- 172.21.15.16:0/2093738925 >> v2:172.21.15.142:3300/0 conn(0x7f34640ca8a0 0x7f34640cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.470+0000 7f3469400640 1 -- 172.21.15.16:0/2093738925 >> v1:172.21.15.142:6789/0 conn(0x7f34640ccad0 legacy=0x7f34640cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.470+0000 7f3469400640 1 -- 172.21.15.16:0/2093738925 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34640cb060 con 0x7f3464159ca0 2024-08-17T02:45:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.471+0000 7f3469400640 1 --2- 172.21.15.16:0/2093738925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3464159ca0 0x7f34640ca360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f346408a060 tx=0x7f34580091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4760ae1f1b38137 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.472+0000 7f3463e00640 1 -- 172.21.15.16:0/2093738925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345802ae80 con 0x7f3464159ca0 2024-08-17T02:45:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.472+0000 7f3463e00640 1 -- 172.21.15.16:0/2093738925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3458032440 con 0x7f3464159ca0 2024-08-17T02:45:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.472+0000 7f3463e00640 1 -- 172.21.15.16:0/2093738925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458032800 con 0x7f3464159ca0 2024-08-17T02:45:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.472+0000 7f346a800640 1 -- 172.21.15.16:0/2093738925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3464159ca0 msgr2=0x7f34640ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.472+0000 7f346a800640 1 --2- 172.21.15.16:0/2093738925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3464159ca0 0x7f34640ca360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f346408a060 tx=0x7f34580091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.472+0000 7f346a800640 1 -- 172.21.15.16:0/2093738925 shutdown_connections 2024-08-17T02:45:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.472+0000 7f346a800640 1 --2- 172.21.15.16:0/2093738925 >> v2:172.21.15.142:3300/0 conn(0x7f34640ca8a0 0x7f34640cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.472+0000 7f346a800640 1 --2- 172.21.15.16:0/2093738925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3464159ca0 0x7f34640ca360 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.472+0000 7f346a800640 1 -- 172.21.15.16:0/2093738925 >> 172.21.15.16:0/2093738925 conn(0x7f3464098e00 msgr2=0x7f346406eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.472+0000 7f346a800640 1 -- 172.21.15.16:0/2093738925 shutdown_connections 2024-08-17T02:45:56.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.472+0000 7f346a800640 1 -- 172.21.15.16:0/2093738925 wait complete. 2024-08-17T02:45:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.474+0000 7f346a800640 1 Processor -- start 2024-08-17T02:45:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.474+0000 7f346a800640 1 -- start start 2024-08-17T02:45:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f346a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34640ca8a0 0x7f34640c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f346a800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34640ccad0 0x7f346415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f346a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464159ca0 0x7f3464155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f346a800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34641563a0 con 0x7f34640ca8a0 2024-08-17T02:45:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f346a800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346415dbf0 con 0x7f3464159ca0 2024-08-17T02:45:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f346a800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34640c91e0 con 0x7f34640ccad0 2024-08-17T02:45:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f3469400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34640ca8a0 0x7f34640c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f3469400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34640ca8a0 0x7f34640c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49362/0 (socket says 172.21.15.16:49362) 2024-08-17T02:45:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f3469400640 1 -- 172.21.15.16:0/1627013633 learned_addr learned my addr 172.21.15.16:0/1627013633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:56.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f3468a00640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34640ccad0 0x7f346415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f3469e00640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464159ca0 0x7f3464155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f3469400640 1 -- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34640ccad0 msgr2=0x7f346415d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f3469400640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34640ccad0 0x7f346415d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f3469400640 1 -- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464159ca0 msgr2=0x7f3464155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.475+0000 7f3469400640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464159ca0 0x7f3464155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.476+0000 7f3469400640 1 -- 172.21.15.16:0/1627013633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3458009450 con 0x7f34640ca8a0 2024-08-17T02:45:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.476+0000 7f3469e00640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464159ca0 0x7f3464155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:56.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.476+0000 7f3469400640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34640ca8a0 0x7f34640c9670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3458009940 tx=0x7f3458032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.476+0000 7f3460c00640 1 -- 172.21.15.16:0/1627013633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458000d10 con 0x7f34640ca8a0 2024-08-17T02:45:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.476+0000 7f346a800640 1 -- 172.21.15.16:0/1627013633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34640c85d0 con 0x7f34640ca8a0 2024-08-17T02:45:56.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.476+0000 7f346a800640 1 -- 172.21.15.16:0/1627013633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34640c8b90 con 0x7f34640ca8a0 2024-08-17T02:45:56.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.477+0000 7f3460c00640 1 -- 172.21.15.16:0/1627013633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3458047440 con 0x7f34640ca8a0 2024-08-17T02:45:56.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.477+0000 7f3460c00640 1 -- 172.21.15.16:0/1627013633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34580477c0 con 0x7f34640ca8a0 2024-08-17T02:45:56.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.477+0000 7f346a800640 1 -- 172.21.15.16:0/1627013633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3440005210 con 0x7f34640ca8a0 2024-08-17T02:45:56.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.478+0000 7f3460c00640 1 -- 172.21.15.16:0/1627013633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f3458039040 con 0x7f34640ca8a0 2024-08-17T02:45:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.479+0000 7f3460c00640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3444065150 0x7f3444067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.479+0000 7f3460c00640 1 -- 172.21.15.16:0/1627013633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f34580a26b0 con 0x7f34640ca8a0 2024-08-17T02:45:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.480+0000 7f3468a00640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3444065150 0x7f3444067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.480+0000 7f3468a00640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3444065150 0x7f3444067600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f346416c010 tx=0x7f3450008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:56.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.487+0000 7f3460c00640 1 -- 172.21.15.16:0/1627013633 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f3458070780 con 0x7f34640ca8a0 2024-08-17T02:45:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.719+0000 7f346a800640 1 -- 172.21.15.16:0/1627013633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3440005a40 con 0x7f34640ca8a0 2024-08-17T02:45:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.720+0000 7f3460c00640 1 -- 172.21.15.16:0/1627013633 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f3458037090 con 0x7f34640ca8a0 2024-08-17T02:45:56.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:56.721 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/111049205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":111049205},{"type":"v1","addr":"172.21.15.142:6836","nonce":111049205}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2692359383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2692359383},{"type":"v1","addr":"172.21.15.142:6837","nonce":2692359383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3179355872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3179355872},{"type":"v1","addr":"172.21.15.16:6833","nonce":3179355872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:45:19.450658+0000","modified":"2024-08-17T02:45:23.039406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2140506421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2140506421},{"type":"v1","addr":"172.21.15.16:6835","nonce":2140506421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:45:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-08-17T02:45:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.722+0000 7f346a800640 1 -- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3444065150 msgr2=0x7f3444067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.722+0000 7f346a800640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3444065150 0x7f3444067600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f346416c010 tx=0x7f3450008040 comp rx=0 tx=0).stop 2024-08-17T02:45:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.722+0000 7f346a800640 1 -- 172.21.15.16:0/1627013633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34640ca8a0 msgr2=0x7f34640c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.722+0000 7f346a800640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34640ca8a0 0x7f34640c9670 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3458009940 tx=0x7f3458032000 comp rx=0 tx=0).stop 2024-08-17T02:45:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.722+0000 7f346a800640 1 -- 172.21.15.16:0/1627013633 shutdown_connections 2024-08-17T02:45:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.722+0000 7f346a800640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3444065150 0x7f3444067600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.722+0000 7f346a800640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3464159ca0 0x7f3464155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.722+0000 7f346a800640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f34640ccad0 0x7f346415d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.722+0000 7f346a800640 1 --2- 172.21.15.16:0/1627013633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34640ca8a0 0x7f34640c9670 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.722+0000 7f346a800640 1 -- 172.21.15.16:0/1627013633 >> 172.21.15.16:0/1627013633 conn(0x7f3464098e00 msgr2=0x7f34640cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.722+0000 7f346a800640 1 -- 172.21.15.16:0/1627013633 shutdown_connections 2024-08-17T02:45:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.722+0000 7f346a800640 1 -- 172.21.15.16:0/1627013633 wait complete. 2024-08-17T02:45:56.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:56.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.889+0000 7f1c63200640 1 Processor -- start 2024-08-17T02:45:56.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.889+0000 7f1c63200640 1 -- start start 2024-08-17T02:45:56.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.890+0000 7f1c63200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1c64155900 0x7f1c64155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.890+0000 7f1c63200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c64158b30 0x7f1c640ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.890+0000 7f1c63200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c64157690 con 0x7f1c640b0270 2024-08-17T02:45:56.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.890+0000 7f1c63200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c64156a80 con 0x7f1c64158b30 2024-08-17T02:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.890+0000 7f1c63200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c64156d60 con 0x7f1c64155900 2024-08-17T02:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.890+0000 7f1c62800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c64158b30 0x7f1c640ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.890+0000 7f1c62800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c64158b30 0x7f1c640ae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49378/0 (socket says 172.21.15.16:49378) 2024-08-17T02:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.890+0000 7f1c62800640 1 -- 172.21.15.16:0/2777683995 learned_addr learned my addr 172.21.15.16:0/2777683995 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.890+0000 7f1c61e00640 1 --2- 172.21.15.16:0/2777683995 >> v2:172.21.15.142:3300/0 conn(0x7f1c64155900 0x7f1c64155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.891+0000 7f1c62800640 1 -- 172.21.15.16:0/2777683995 >> v2:172.21.15.142:3300/0 conn(0x7f1c64155900 msgr2=0x7f1c64155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:56.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.891+0000 7f1c62800640 1 --2- 172.21.15.16:0/2777683995 >> v2:172.21.15.142:3300/0 conn(0x7f1c64155900 0x7f1c64155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.891+0000 7f1c62800640 1 -- 172.21.15.16:0/2777683995 >> v1:172.21.15.142:6789/0 conn(0x7f1c640b0270 legacy=0x7f1c64156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:56.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.891+0000 7f1c62800640 1 -- 172.21.15.16:0/2777683995 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c64157040 con 0x7f1c64158b30 2024-08-17T02:45:56.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.892+0000 7f1c62800640 1 --2- 172.21.15.16:0/2777683995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c64158b30 0x7f1c640ae6b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1c64065fd0 tx=0x7f1c5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d880cb9afd6e9a5f server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:56.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.892+0000 7f1c61400640 1 -- 172.21.15.16:0/2777683995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c02e050 con 0x7f1c64158b30 2024-08-17T02:45:56.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.892+0000 7f1c61400640 1 -- 172.21.15.16:0/2777683995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c007400 con 0x7f1c64158b30 2024-08-17T02:45:56.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.892+0000 7f1c61400640 1 -- 172.21.15.16:0/2777683995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0077e0 con 0x7f1c64158b30 2024-08-17T02:45:56.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.892+0000 7f1c63200640 1 -- 172.21.15.16:0/2777683995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c64158b30 msgr2=0x7f1c640ae6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.892+0000 7f1c63200640 1 --2- 172.21.15.16:0/2777683995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c64158b30 0x7f1c640ae6b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1c64065fd0 tx=0x7f1c5c009840 comp rx=0 tx=0).stop 2024-08-17T02:45:56.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.893+0000 7f1c63200640 1 -- 172.21.15.16:0/2777683995 shutdown_connections 2024-08-17T02:45:56.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.893+0000 7f1c63200640 1 --2- 172.21.15.16:0/2777683995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c64158b30 0x7f1c640ae6b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.893+0000 7f1c63200640 1 --2- 172.21.15.16:0/2777683995 >> v2:172.21.15.142:3300/0 conn(0x7f1c64155900 0x7f1c64155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.893+0000 7f1c63200640 1 -- 172.21.15.16:0/2777683995 >> 172.21.15.16:0/2777683995 conn(0x7f1c64067750 msgr2=0x7f1c640cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.893+0000 7f1c63200640 1 -- 172.21.15.16:0/2777683995 shutdown_connections 2024-08-17T02:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.893+0000 7f1c63200640 1 -- 172.21.15.16:0/2777683995 wait complete. 2024-08-17T02:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.894+0000 7f1c63200640 1 Processor -- start 2024-08-17T02:45:56.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.894+0000 7f1c63200640 1 -- start start 2024-08-17T02:45:56.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.895+0000 7f1c63200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c640b0270 0x7f1c64148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.895+0000 7f1c63200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c64155900 0x7f1c64168cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.895+0000 7f1c63200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c64158b30 0x7f1c64152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.895+0000 7f1c63200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640c91a0 con 0x7f1c64158b30 2024-08-17T02:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.895+0000 7f1c63200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c64169640 con 0x7f1c64155900 2024-08-17T02:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.895+0000 7f1c63200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c64169920 con 0x7f1c640b0270 2024-08-17T02:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.896+0000 7f1c62800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c64158b30 0x7f1c64152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.896+0000 7f1c62800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c64158b30 0x7f1c64152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49382/0 (socket says 172.21.15.16:49382) 2024-08-17T02:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.896+0000 7f1c62800640 1 -- 172.21.15.16:0/3717607938 learned_addr learned my addr 172.21.15.16:0/3717607938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.896+0000 7f1c61e00640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c640b0270 0x7f1c64148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.896+0000 7f1c59400640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c64155900 0x7f1c64168cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.896+0000 7f1c62800640 1 -- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c640b0270 msgr2=0x7f1c64148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.896+0000 7f1c62800640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c640b0270 0x7f1c64148130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.896+0000 7f1c62800640 1 -- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c64155900 msgr2=0x7f1c64168cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:56.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.896+0000 7f1c62800640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c64155900 0x7f1c64168cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:56.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.896+0000 7f1c62800640 1 -- 172.21.15.16:0/3717607938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c009ae0 con 0x7f1c64158b30 2024-08-17T02:45:56.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.896+0000 7f1c62800640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c64158b30 0x7f1c64152d60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009f50 tx=0x7f1c5c037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:56.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.897+0000 7f1c5a000640 1 -- 172.21.15.16:0/3717607938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c033e40 con 0x7f1c64158b30 2024-08-17T02:45:56.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.897+0000 7f1c5a000640 1 -- 172.21.15.16:0/3717607938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c03c410 con 0x7f1c64158b30 2024-08-17T02:45:56.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.897+0000 7f1c5a000640 1 -- 172.21.15.16:0/3717607938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c03c7d0 con 0x7f1c64158b30 2024-08-17T02:45:56.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.897+0000 7f1c63200640 1 -- 172.21.15.16:0/3717607938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c640c9440 con 0x7f1c64158b30 2024-08-17T02:45:56.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.897+0000 7f1c63200640 1 -- 172.21.15.16:0/3717607938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c64169ea0 con 0x7f1c64158b30 2024-08-17T02:45:56.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.898+0000 7f1c63200640 1 -- 172.21.15.16:0/3717607938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c64071050 con 0x7f1c64158b30 2024-08-17T02:45:56.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.899+0000 7f1c5a000640 1 -- 172.21.15.16:0/3717607938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1c5c039050 con 0x7f1c64158b30 2024-08-17T02:45:56.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.900+0000 7f1c5a000640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1c40065150 0x7f1c40067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.900+0000 7f1c5a000640 1 -- 172.21.15.16:0/3717607938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1c5c0a2e90 con 0x7f1c64158b30 2024-08-17T02:45:56.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.900+0000 7f1c61e00640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1c40065150 0x7f1c40067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:56.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.901+0000 7f1c61e00640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1c40065150 0x7f1c40067600 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1c640afae0 tx=0x7f1c54008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:56.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:56.904+0000 7f1c5a000640 1 -- 172.21.15.16:0/3717607938 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1c5c071000 con 0x7f1c64158b30 2024-08-17T02:45:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.088+0000 7f1c63200640 1 -- 172.21.15.16:0/3717607938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c6406f480 con 0x7f1c64158b30 2024-08-17T02:45:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.089+0000 7f1c5a000640 1 -- 172.21.15.16:0/3717607938 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f1c5c074620 con 0x7f1c64158b30 2024-08-17T02:45:57.090 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:57.091 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.091+0000 7f1c63200640 1 -- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1c40065150 msgr2=0x7f1c40067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.091+0000 7f1c63200640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1c40065150 0x7f1c40067600 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1c640afae0 tx=0x7f1c54008040 comp rx=0 tx=0).stop 2024-08-17T02:45:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.091+0000 7f1c63200640 1 -- 172.21.15.16:0/3717607938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c64158b30 msgr2=0x7f1c64152d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.091+0000 7f1c63200640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c64158b30 0x7f1c64152d60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009f50 tx=0x7f1c5c037000 comp rx=0 tx=0).stop 2024-08-17T02:45:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.091+0000 7f1c63200640 1 -- 172.21.15.16:0/3717607938 shutdown_connections 2024-08-17T02:45:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.091+0000 7f1c63200640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1c40065150 0x7f1c40067600 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.091+0000 7f1c63200640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c64158b30 0x7f1c64152d60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.091+0000 7f1c63200640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c64155900 0x7f1c64168cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.091+0000 7f1c63200640 1 --2- 172.21.15.16:0/3717607938 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c640b0270 0x7f1c64148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.091+0000 7f1c63200640 1 -- 172.21.15.16:0/3717607938 >> 172.21.15.16:0/3717607938 conn(0x7f1c64067750 msgr2=0x7f1c64068690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.092+0000 7f1c63200640 1 -- 172.21.15.16:0/3717607938 shutdown_connections 2024-08-17T02:45:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.092+0000 7f1c63200640 1 -- 172.21.15.16:0/3717607938 wait complete. 2024-08-17T02:45:57.109 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-17T02:45:57.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:45:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.260+0000 7f2753200640 1 Processor -- start 2024-08-17T02:45:57.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.261+0000 7f2753200640 1 -- start start 2024-08-17T02:45:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.262+0000 7f2753200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f274c156ad0 0x7f274c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.262+0000 7f2753200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f274c156ea0 0x7f274c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.262+0000 7f2753200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f274c06dab0 0x7f274c06df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.262+0000 7f2753200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274c06e450 con 0x7f274c156ad0 2024-08-17T02:45:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.262+0000 7f2753200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274c0ca900 con 0x7f274c06dab0 2024-08-17T02:45:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.262+0000 7f2753200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0cc560 con 0x7f274c156ea0 2024-08-17T02:45:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.262+0000 7f2751e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f274c156ad0 0x7f274c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.262+0000 7f2751e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f274c156ad0 0x7f274c0cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49384/0 (socket says 172.21.15.16:49384) 2024-08-17T02:45:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.262+0000 7f2751e00640 1 -- 172.21.15.16:0/1048451757 learned_addr learned my addr 172.21.15.16:0/1048451757 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.262+0000 7f2751400640 1 --2- 172.21.15.16:0/1048451757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f274c156ea0 0x7f274c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:57.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.262+0000 7f2752800640 1 --2- 172.21.15.16:0/1048451757 >> v2:172.21.15.142:3300/0 conn(0x7f274c06dab0 0x7f274c06df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.263+0000 7f2751e00640 1 -- 172.21.15.16:0/1048451757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f274c156ea0 msgr2=0x7f274c06d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.263+0000 7f2751e00640 1 --2- 172.21.15.16:0/1048451757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f274c156ea0 0x7f274c06d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.263+0000 7f2751e00640 1 -- 172.21.15.16:0/1048451757 >> v2:172.21.15.142:3300/0 conn(0x7f274c06dab0 msgr2=0x7f274c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.263+0000 7f2751e00640 1 --2- 172.21.15.16:0/1048451757 >> v2:172.21.15.142:3300/0 conn(0x7f274c06dab0 0x7f274c06df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.263+0000 7f2751e00640 1 -- 172.21.15.16:0/1048451757 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c0caaf0 con 0x7f274c156ad0 2024-08-17T02:45:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.264+0000 7f2751e00640 1 --2- 172.21.15.16:0/1048451757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c156ad0 0x7f274c0cbea0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f274c0cb740 tx=0x7f27400091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7dc16b545750e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.264+0000 7f2750a00640 1 -- 172.21.15.16:0/1048451757 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274002ade0 con 0x7f274c156ad0 2024-08-17T02:45:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.264+0000 7f2750a00640 1 -- 172.21.15.16:0/1048451757 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2740032440 con 0x7f274c156ad0 2024-08-17T02:45:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.264+0000 7f2750a00640 1 -- 172.21.15.16:0/1048451757 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27400327e0 con 0x7f274c156ad0 2024-08-17T02:45:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.264+0000 7f2753200640 1 -- 172.21.15.16:0/1048451757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c156ad0 msgr2=0x7f274c0cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.264+0000 7f2753200640 1 --2- 172.21.15.16:0/1048451757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c156ad0 0x7f274c0cbea0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f274c0cb740 tx=0x7f27400091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.265+0000 7f2753200640 1 -- 172.21.15.16:0/1048451757 shutdown_connections 2024-08-17T02:45:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.265+0000 7f2753200640 1 --2- 172.21.15.16:0/1048451757 >> v2:172.21.15.142:3300/0 conn(0x7f274c06dab0 0x7f274c06df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.265+0000 7f2753200640 1 --2- 172.21.15.16:0/1048451757 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f274c156ea0 0x7f274c06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.265+0000 7f2753200640 1 --2- 172.21.15.16:0/1048451757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c156ad0 0x7f274c0cbea0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.265+0000 7f2753200640 1 -- 172.21.15.16:0/1048451757 >> 172.21.15.16:0/1048451757 conn(0x7f274c067750 msgr2=0x7f274c0cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.265+0000 7f2753200640 1 -- 172.21.15.16:0/1048451757 shutdown_connections 2024-08-17T02:45:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.265+0000 7f2753200640 1 -- 172.21.15.16:0/1048451757 wait complete. 2024-08-17T02:45:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.266+0000 7f2753200640 1 Processor -- start 2024-08-17T02:45:57.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.266+0000 7f2753200640 1 -- start start 2024-08-17T02:45:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.267+0000 7f2753200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c06dab0 0x7f274c0c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:57.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.267+0000 7f2753200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f274c156ad0 0x7f274c13e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.267+0000 7f2753200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f274c156ea0 0x7f274c15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.267+0000 7f2753200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0c0a30 con 0x7f274c06dab0 2024-08-17T02:45:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.267+0000 7f2753200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0c6680 con 0x7f274c156ea0 2024-08-17T02:45:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.267+0000 7f2753200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0c5a70 con 0x7f274c156ad0 2024-08-17T02:45:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.267+0000 7f2751e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c06dab0 0x7f274c0c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.267+0000 7f2751e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c06dab0 0x7f274c0c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49400/0 (socket says 172.21.15.16:49400) 2024-08-17T02:45:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.267+0000 7f2751e00640 1 -- 172.21.15.16:0/1832846767 learned_addr learned my addr 172.21.15.16:0/1832846767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.267+0000 7f2751400640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f274c156ad0 0x7f274c13e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.267+0000 7f2752800640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f274c156ea0 0x7f274c15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.268+0000 7f2751e00640 1 -- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f274c156ad0 msgr2=0x7f274c13e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.268+0000 7f2751e00640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f274c156ad0 0x7f274c13e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.268+0000 7f2751e00640 1 -- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f274c156ea0 msgr2=0x7f274c15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.268+0000 7f2751e00640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f274c156ea0 0x7f274c15ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.268+0000 7f2751e00640 1 -- 172.21.15.16:0/1832846767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2740009450 con 0x7f274c06dab0 2024-08-17T02:45:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.268+0000 7f2751e00640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c06dab0 0x7f274c0c5440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2740035040 tx=0x7f2740032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.268+0000 7f2745600640 1 -- 172.21.15.16:0/1832846767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740031c80 con 0x7f274c06dab0 2024-08-17T02:45:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.268+0000 7f2745600640 1 -- 172.21.15.16:0/1832846767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2740031e20 con 0x7f274c06dab0 2024-08-17T02:45:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.268+0000 7f2753200640 1 -- 172.21.15.16:0/1832846767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274c0c5d50 con 0x7f274c06dab0 2024-08-17T02:45:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.268+0000 7f2745600640 1 -- 172.21.15.16:0/1832846767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274003d670 con 0x7f274c06dab0 2024-08-17T02:45:57.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.268+0000 7f2753200640 1 -- 172.21.15.16:0/1832846767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274c0c6390 con 0x7f274c06dab0 2024-08-17T02:45:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.270+0000 7f2753200640 1 -- 172.21.15.16:0/1832846767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274c071050 con 0x7f274c06dab0 2024-08-17T02:45:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.270+0000 7f2745600640 1 -- 172.21.15.16:0/1832846767 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f2740039040 con 0x7f274c06dab0 2024-08-17T02:45:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.271+0000 7f2745600640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2730065150 0x7f2730067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.271+0000 7f2745600640 1 -- 172.21.15.16:0/1832846767 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f27400a2730 con 0x7f274c06dab0 2024-08-17T02:45:57.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.272+0000 7f2751400640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2730065150 0x7f2730067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:57.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.272+0000 7f2751400640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2730065150 0x7f2730067600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f274c1457a0 tx=0x7f273c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:57.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.275+0000 7f2745600640 1 -- 172.21.15.16:0/1832846767 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2740070750 con 0x7f274c06dab0 2024-08-17T02:45:57.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.495+0000 7f2753200640 1 -- 172.21.15.16:0/1832846767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f274c06f480 con 0x7f274c06dab0 2024-08-17T02:45:57.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.496+0000 7f2745600640 1 -- 172.21.15.16:0/1832846767 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f2740073d70 con 0x7f274c06dab0 2024-08-17T02:45:57.497 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:57.497 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/111049205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":111049205},{"type":"v1","addr":"172.21.15.142:6836","nonce":111049205}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2692359383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2692359383},{"type":"v1","addr":"172.21.15.142:6837","nonce":2692359383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3179355872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3179355872},{"type":"v1","addr":"172.21.15.16:6833","nonce":3179355872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:45:19.450658+0000","modified":"2024-08-17T02:45:23.039406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2140506421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2140506421},{"type":"v1","addr":"172.21.15.16:6835","nonce":2140506421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:45:57.497 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-08-17T02:45:57.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.498+0000 7f2753200640 1 -- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2730065150 msgr2=0x7f2730067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.498+0000 7f2753200640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2730065150 0x7f2730067600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f274c1457a0 tx=0x7f273c008040 comp rx=0 tx=0).stop 2024-08-17T02:45:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.498+0000 7f2753200640 1 -- 172.21.15.16:0/1832846767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c06dab0 msgr2=0x7f274c0c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.498+0000 7f2753200640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c06dab0 0x7f274c0c5440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2740035040 tx=0x7f2740032000 comp rx=0 tx=0).stop 2024-08-17T02:45:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.498+0000 7f2753200640 1 -- 172.21.15.16:0/1832846767 shutdown_connections 2024-08-17T02:45:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.498+0000 7f2753200640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2730065150 0x7f2730067600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.498+0000 7f2753200640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f274c156ea0 0x7f274c15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.498+0000 7f2753200640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f274c156ad0 0x7f274c13e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.498+0000 7f2753200640 1 --2- 172.21.15.16:0/1832846767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f274c06dab0 0x7f274c0c5440 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.498+0000 7f2753200640 1 -- 172.21.15.16:0/1832846767 >> 172.21.15.16:0/1832846767 conn(0x7f274c067750 msgr2=0x7f274c0c8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.498+0000 7f2753200640 1 -- 172.21.15.16:0/1832846767 shutdown_connections 2024-08-17T02:45:57.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.498+0000 7f2753200640 1 -- 172.21.15.16:0/1832846767 wait complete. 2024-08-17T02:45:57.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:45:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.664+0000 7f51f5400640 1 Processor -- start 2024-08-17T02:45:57.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.664+0000 7f51f5400640 1 -- start start 2024-08-17T02:45:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.666+0000 7f51f5400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f51f015c7c0 0x7f51f015cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.666+0000 7f51f5400640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f00ab950 con 0x7f51f00ab2b0 2024-08-17T02:45:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.666+0000 7f51f5400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f00abc30 con 0x7f51f00cc010 2024-08-17T02:45:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.666+0000 7f51f5400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f00ab730 con 0x7f51f015c7c0 2024-08-17T02:45:57.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.666+0000 7f51e7e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f51f00ab2b0 0x7f51f00afe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56202/0 (socket says 172.21.15.16:56202) 2024-08-17T02:45:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.666+0000 7f51e7e00640 1 -- 172.21.15.16:0/2662978977 learned_addr learned my addr 172.21.15.16:0/2662978977 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.666+0000 7f51f4a00640 1 --2- 172.21.15.16:0/2662978977 >> v2:172.21.15.142:3300/0 conn(0x7f51f015c7c0 0x7f51f015cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.667+0000 7f51ef400640 1 -- 172.21.15.16:0/2662978977 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113388949 0 0) 0x7f51f00ab950 con 0x7f51f00ab2b0 2024-08-17T02:45:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.667+0000 7f51ef400640 1 -- 172.21.15.16:0/2662978977 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d40034e0 con 0x7f51f00ab2b0 2024-08-17T02:45:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.667+0000 7f51f4a00640 1 -- 172.21.15.16:0/2662978977 >> v1:172.21.15.142:6789/0 conn(0x7f51f00cc010 legacy=0x7f51f00a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.667+0000 7f51f4a00640 1 -- 172.21.15.16:0/2662978977 >> v1:172.21.15.16:6789/0 conn(0x7f51f00ab2b0 legacy=0x7f51f00afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.667+0000 7f51f4a00640 1 -- 172.21.15.16:0/2662978977 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f00abf10 con 0x7f51f015c7c0 2024-08-17T02:45:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.668+0000 7f51f4a00640 1 --2- 172.21.15.16:0/2662978977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f015c7c0 0x7f51f015cc00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009e60 tx=0x7f51e8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=798013cba928e1d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.668+0000 7f51ef400640 1 -- 172.21.15.16:0/2662978977 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e8007550 con 0x7f51f015c7c0 2024-08-17T02:45:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.668+0000 7f51ef400640 1 -- 172.21.15.16:0/2662978977 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e802d050 con 0x7f51f015c7c0 2024-08-17T02:45:57.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.668+0000 7f51ef400640 1 -- 172.21.15.16:0/2662978977 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e80078c0 con 0x7f51f015c7c0 2024-08-17T02:45:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.669+0000 7f51f5400640 1 -- 172.21.15.16:0/2662978977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f015c7c0 msgr2=0x7f51f015cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.669+0000 7f51f5400640 1 --2- 172.21.15.16:0/2662978977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f015c7c0 0x7f51f015cc00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009e60 tx=0x7f51e8009780 comp rx=0 tx=0).stop 2024-08-17T02:45:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.669+0000 7f51f5400640 1 -- 172.21.15.16:0/2662978977 shutdown_connections 2024-08-17T02:45:57.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.669+0000 7f51f5400640 1 --2- 172.21.15.16:0/2662978977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f015c7c0 0x7f51f015cc00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.669+0000 7f51f5400640 1 -- 172.21.15.16:0/2662978977 >> 172.21.15.16:0/2662978977 conn(0x7f51f0067de0 msgr2=0x7f51f00681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.669+0000 7f51f5400640 1 -- 172.21.15.16:0/2662978977 shutdown_connections 2024-08-17T02:45:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.669+0000 7f51f5400640 1 -- 172.21.15.16:0/2662978977 wait complete. 2024-08-17T02:45:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.670+0000 7f51f5400640 1 Processor -- start 2024-08-17T02:45:57.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.670+0000 7f51f5400640 1 -- start start 2024-08-17T02:45:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.671+0000 7f51f5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00ab2b0 0x7f51f00a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.671+0000 7f51f5400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51f00cc010 0x7f51f00ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.671+0000 7f51f5400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f015c7c0 0x7f51f014bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.671+0000 7f51f5400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f00a8220 con 0x7f51f00ab2b0 2024-08-17T02:45:57.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.671+0000 7f51f5400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f016d940 con 0x7f51f015c7c0 2024-08-17T02:45:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.671+0000 7f51f5400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f016dc20 con 0x7f51f00cc010 2024-08-17T02:45:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.671+0000 7f51efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00ab2b0 0x7f51f00a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.672+0000 7f51efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00ab2b0 0x7f51f00a54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49408/0 (socket says 172.21.15.16:49408) 2024-08-17T02:45:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.672+0000 7f51efe00640 1 -- 172.21.15.16:0/818305187 learned_addr learned my addr 172.21.15.16:0/818305187 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.672+0000 7f51f4a00640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f015c7c0 0x7f51f014bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.672+0000 7f51e7e00640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51f00cc010 0x7f51f00ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.672+0000 7f51efe00640 1 -- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51f00cc010 msgr2=0x7f51f00ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.672+0000 7f51efe00640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51f00cc010 0x7f51f00ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.672+0000 7f51efe00640 1 -- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f015c7c0 msgr2=0x7f51f014bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.672+0000 7f51efe00640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f015c7c0 0x7f51f014bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.672+0000 7f51efe00640 1 -- 172.21.15.16:0/818305187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e8009a20 con 0x7f51f00ab2b0 2024-08-17T02:45:57.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.672+0000 7f51efe00640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00ab2b0 0x7f51f00a54d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f51e000c560 tx=0x7f51e0007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.672+0000 7f51e7400640 1 -- 172.21.15.16:0/818305187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0011590 con 0x7f51f00ab2b0 2024-08-17T02:45:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.673+0000 7f51e7400640 1 -- 172.21.15.16:0/818305187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0011710 con 0x7f51f00ab2b0 2024-08-17T02:45:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.673+0000 7f51f5400640 1 -- 172.21.15.16:0/818305187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f016df00 con 0x7f51f00ab2b0 2024-08-17T02:45:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.673+0000 7f51e7400640 1 -- 172.21.15.16:0/818305187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0011ad0 con 0x7f51f00ab2b0 2024-08-17T02:45:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.673+0000 7f51f5400640 1 -- 172.21.15.16:0/818305187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f00a5a10 con 0x7f51f00ab2b0 2024-08-17T02:45:57.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.674+0000 7f51f5400640 1 -- 172.21.15.16:0/818305187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f0071050 con 0x7f51f00ab2b0 2024-08-17T02:45:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.675+0000 7f51e7400640 1 -- 172.21.15.16:0/818305187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f51e001b3b0 con 0x7f51f00ab2b0 2024-08-17T02:45:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.675+0000 7f51e7400640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f51cc065150 0x7f51cc067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.675+0000 7f51e7400640 1 -- 172.21.15.16:0/818305187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f51e0081a90 con 0x7f51f00ab2b0 2024-08-17T02:45:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.676+0000 7f51e7e00640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f51cc065150 0x7f51cc067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:57.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.676+0000 7f51e7e00640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f51cc065150 0x7f51cc067600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51f0156480 tx=0x7f51d8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:57.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.679+0000 7f51e7400640 1 -- 172.21.15.16:0/818305187 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f51e004fab0 con 0x7f51f00ab2b0 2024-08-17T02:45:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.904+0000 7f51f5400640 1 -- 172.21.15.16:0/818305187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51f00cc3e0 con 0x7f51f00ab2b0 2024-08-17T02:45:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.905+0000 7f51e7400640 1 -- 172.21.15.16:0/818305187 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4594 (secure 0 0 0) 0x7f51e0017070 con 0x7f51f00ab2b0 2024-08-17T02:45:57.907 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:57.907 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6836/111049205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":111049205},{"type":"v1","addr":"172.21.15.142:6836","nonce":111049205}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4255,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2692359383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6835","nonce":2692359383},{"type":"v1","addr":"172.21.15.142:6837","nonce":2692359383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/3179355872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3179355872},{"type":"v1","addr":"172.21.15.16:6833","nonce":3179355872}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:45:19.450658+0000","modified":"2024-08-17T02:45:23.039406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4266},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4266":{"gid":4266,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6835/2140506421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":2140506421},{"type":"v1","addr":"172.21.15.16:6835","nonce":2140506421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2024-08-17T02:45:57.907 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2024-08-17T02:45:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.907+0000 7f51f5400640 1 -- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f51cc065150 msgr2=0x7f51cc067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.907+0000 7f51f5400640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f51cc065150 0x7f51cc067600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51f0156480 tx=0x7f51d8009040 comp rx=0 tx=0).stop 2024-08-17T02:45:57.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.907+0000 7f51f5400640 1 -- 172.21.15.16:0/818305187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00ab2b0 msgr2=0x7f51f00a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:57.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.907+0000 7f51f5400640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00ab2b0 0x7f51f00a54d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f51e000c560 tx=0x7f51e0007ef0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.907+0000 7f51f5400640 1 -- 172.21.15.16:0/818305187 shutdown_connections 2024-08-17T02:45:57.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.907+0000 7f51f5400640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f51cc065150 0x7f51cc067600 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.907+0000 7f51f5400640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51f015c7c0 0x7f51f014bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.907+0000 7f51f5400640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f51f00cc010 0x7f51f00ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.907+0000 7f51f5400640 1 --2- 172.21.15.16:0/818305187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51f00ab2b0 0x7f51f00a54d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.907+0000 7f51f5400640 1 -- 172.21.15.16:0/818305187 >> 172.21.15.16:0/818305187 conn(0x7f51f0067de0 msgr2=0x7f51f00ae290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.907+0000 7f51f5400640 1 -- 172.21.15.16:0/818305187 shutdown_connections 2024-08-17T02:45:57.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:57.907+0000 7f51f5400640 1 -- 172.21.15.16:0/818305187 wait complete. 2024-08-17T02:45:57.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:45:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.074+0000 7f90a9400640 1 Processor -- start 2024-08-17T02:45:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.075+0000 7f90a9400640 1 -- start start 2024-08-17T02:45:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.076+0000 7f90a9400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90a4158ad0 0x7f90a40ac070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.076+0000 7f90a9400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f90a40ac5b0 0x7f90a40afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.076+0000 7f90a9400640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a4067d60 con 0x7f90a4158ea0 2024-08-17T02:45:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.076+0000 7f90a9400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a40aca30 con 0x7f90a4158ad0 2024-08-17T02:45:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.076+0000 7f90a9400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a4068600 con 0x7f90a40ac5b0 2024-08-17T02:45:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.076+0000 7f90a3e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90a4158ad0 0x7f90a40ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.076+0000 7f909be00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f90a4158ea0 0x7f90a40af410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56212/0 (socket says 172.21.15.16:56212) 2024-08-17T02:45:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.076+0000 7f909be00640 1 -- 172.21.15.16:0/652251172 learned_addr learned my addr 172.21.15.16:0/652251172 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.076+0000 7f90a8a00640 1 --2- 172.21.15.16:0/652251172 >> v2:172.21.15.142:3300/0 conn(0x7f90a40ac5b0 0x7f90a40afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.077+0000 7f90a3400640 1 -- 172.21.15.16:0/652251172 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972612244 0 0) 0x7f90a4067d60 con 0x7f90a4158ea0 2024-08-17T02:45:58.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.077+0000 7f90a3400640 1 -- 172.21.15.16:0/652251172 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90880034e0 con 0x7f90a4158ea0 2024-08-17T02:45:58.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.077+0000 7f90a3e00640 1 -- 172.21.15.16:0/652251172 >> v2:172.21.15.142:3300/0 conn(0x7f90a40ac5b0 msgr2=0x7f90a40afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:58.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.077+0000 7f90a3e00640 1 --2- 172.21.15.16:0/652251172 >> v2:172.21.15.142:3300/0 conn(0x7f90a40ac5b0 0x7f90a40afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.077+0000 7f90a3e00640 1 -- 172.21.15.16:0/652251172 >> v1:172.21.15.16:6789/0 conn(0x7f90a4158ea0 legacy=0x7f90a40af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.077+0000 7f90a3e00640 1 -- 172.21.15.16:0/652251172 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a40b01d0 con 0x7f90a4158ad0 2024-08-17T02:45:58.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.077+0000 7f90a8a00640 1 --2- 172.21.15.16:0/652251172 >> v2:172.21.15.142:3300/0 conn(0x7f90a40ac5b0 0x7f90a40afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.078+0000 7f90a3e00640 1 --2- 172.21.15.16:0/652251172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a4158ad0 0x7f90a40ac070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f90a40adbf0 tx=0x7f90940090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be95b98035373154 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.078+0000 7f90a3400640 1 -- 172.21.15.16:0/652251172 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909402ac80 con 0x7f90a4158ad0 2024-08-17T02:45:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.078+0000 7f90a3400640 1 -- 172.21.15.16:0/652251172 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909402ade0 con 0x7f90a4158ad0 2024-08-17T02:45:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.078+0000 7f90a3400640 1 -- 172.21.15.16:0/652251172 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094032670 con 0x7f90a4158ad0 2024-08-17T02:45:58.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.079+0000 7f90a9400640 1 -- 172.21.15.16:0/652251172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a4158ad0 msgr2=0x7f90a40ac070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.079+0000 7f90a9400640 1 --2- 172.21.15.16:0/652251172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a4158ad0 0x7f90a40ac070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f90a40adbf0 tx=0x7f90940090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.079+0000 7f90a9400640 1 -- 172.21.15.16:0/652251172 shutdown_connections 2024-08-17T02:45:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.079+0000 7f90a9400640 1 --2- 172.21.15.16:0/652251172 >> v2:172.21.15.142:3300/0 conn(0x7f90a40ac5b0 0x7f90a40afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.079+0000 7f90a9400640 1 --2- 172.21.15.16:0/652251172 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a4158ad0 0x7f90a40ac070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.079+0000 7f90a9400640 1 -- 172.21.15.16:0/652251172 >> 172.21.15.16:0/652251172 conn(0x7f90a4096a70 msgr2=0x7f90a4096e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.079+0000 7f90a9400640 1 -- 172.21.15.16:0/652251172 shutdown_connections 2024-08-17T02:45:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.079+0000 7f90a9400640 1 -- 172.21.15.16:0/652251172 wait complete. 2024-08-17T02:45:58.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.080+0000 7f90a9400640 1 Processor -- start 2024-08-17T02:45:58.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.080+0000 7f90a9400640 1 -- start start 2024-08-17T02:45:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.081+0000 7f90a9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a40ac5b0 0x7f90a41459a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.081+0000 7f90a9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a4158ad0 0x7f90a4157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.081+0000 7f90a9400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90a4158ea0 0x7f90a4157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.081+0000 7f90a9400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40ccd80 con 0x7f90a4158ad0 2024-08-17T02:45:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.081+0000 7f90a9400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4145ee0 con 0x7f90a40ac5b0 2024-08-17T02:45:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.081+0000 7f90a9400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40b1a30 con 0x7f90a4158ea0 2024-08-17T02:45:58.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.081+0000 7f909be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a4158ad0 0x7f90a4157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.082+0000 7f909be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a4158ad0 0x7f90a4157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49426/0 (socket says 172.21.15.16:49426) 2024-08-17T02:45:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.082+0000 7f909be00640 1 -- 172.21.15.16:0/3135191683 learned_addr learned my addr 172.21.15.16:0/3135191683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.082+0000 7f90a8a00640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90a4158ea0 0x7f90a4157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.082+0000 7f90a3e00640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a40ac5b0 0x7f90a41459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.082+0000 7f909be00640 1 -- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90a4158ea0 msgr2=0x7f90a4157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.082+0000 7f909be00640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90a4158ea0 0x7f90a4157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.082+0000 7f909be00640 1 -- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a40ac5b0 msgr2=0x7f90a41459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.082+0000 7f909be00640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a40ac5b0 0x7f90a41459a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.082+0000 7f909be00640 1 -- 172.21.15.16:0/3135191683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094009390 con 0x7f90a4158ad0 2024-08-17T02:45:58.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.082+0000 7f90a8a00640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90a4158ea0 0x7f90a4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:45:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.082+0000 7f909be00640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a4158ad0 0x7f90a4157010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f908c00c5c0 tx=0x7f908c007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.083+0000 7f909b400640 1 -- 172.21.15.16:0/3135191683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c005aa0 con 0x7f90a4158ad0 2024-08-17T02:45:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.083+0000 7f909b400640 1 -- 172.21.15.16:0/3135191683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c005c00 con 0x7f90a4158ad0 2024-08-17T02:45:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.083+0000 7f90a9400640 1 -- 172.21.15.16:0/3135191683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a40cc170 con 0x7f90a4158ad0 2024-08-17T02:45:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.083+0000 7f909b400640 1 -- 172.21.15.16:0/3135191683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c0070d0 con 0x7f90a4158ad0 2024-08-17T02:45:58.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.083+0000 7f90a9400640 1 -- 172.21.15.16:0/3135191683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a40cc730 con 0x7f90a4158ad0 2024-08-17T02:45:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.084+0000 7f90a9400640 1 -- 172.21.15.16:0/3135191683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a4071050 con 0x7f90a4158ad0 2024-08-17T02:45:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.085+0000 7f909b400640 1 -- 172.21.15.16:0/3135191683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f908c0074c0 con 0x7f90a4158ad0 2024-08-17T02:45:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.085+0000 7f909b400640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9080065150 0x7f9080067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.086+0000 7f909b400640 1 -- 172.21.15.16:0/3135191683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f908c081880 con 0x7f90a4158ad0 2024-08-17T02:45:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.086+0000 7f90a3e00640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9080065150 0x7f9080067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.087+0000 7f90a3e00640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9080065150 0x7f9080067600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9094034000 tx=0x7f9094032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.089+0000 7f909b400640 1 -- 172.21.15.16:0/3135191683 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f908c04f9f0 con 0x7f90a4158ad0 2024-08-17T02:45:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.272+0000 7f90a9400640 1 -- 172.21.15.16:0/3135191683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90a4069a50 con 0x7f90a4158ad0 2024-08-17T02:45:58.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.273+0000 7f909b400640 1 -- 172.21.15.16:0/3135191683 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f908c017020 con 0x7f90a4158ad0 2024-08-17T02:45:58.275 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:45:58.275 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:45:21.011697+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:45:18.004797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-17T02:45:18.430718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:45:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.275+0000 7f90a9400640 1 -- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9080065150 msgr2=0x7f9080067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.275+0000 7f90a9400640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9080065150 0x7f9080067600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9094034000 tx=0x7f9094032000 comp rx=0 tx=0).stop 2024-08-17T02:45:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.275+0000 7f90a9400640 1 -- 172.21.15.16:0/3135191683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a4158ad0 msgr2=0x7f90a4157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.275+0000 7f90a9400640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a4158ad0 0x7f90a4157010 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f908c00c5c0 tx=0x7f908c007d00 comp rx=0 tx=0).stop 2024-08-17T02:45:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.275+0000 7f90a9400640 1 -- 172.21.15.16:0/3135191683 shutdown_connections 2024-08-17T02:45:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.275+0000 7f90a9400640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9080065150 0x7f9080067600 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.275+0000 7f90a9400640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90a4158ea0 0x7f90a4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.275+0000 7f90a9400640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90a4158ad0 0x7f90a4157010 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.275+0000 7f90a9400640 1 --2- 172.21.15.16:0/3135191683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90a40ac5b0 0x7f90a41459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.276+0000 7f90a9400640 1 -- 172.21.15.16:0/3135191683 >> 172.21.15.16:0/3135191683 conn(0x7f90a4096a70 msgr2=0x7f90a40af610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:58.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.276+0000 7f90a9400640 1 -- 172.21.15.16:0/3135191683 shutdown_connections 2024-08-17T02:45:58.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.276+0000 7f90a9400640 1 -- 172.21.15.16:0/3135191683 wait complete. 2024-08-17T02:45:58.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-08-17T02:45:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.447+0000 7f5a61e00640 1 Processor -- start 2024-08-17T02:45:58.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.448+0000 7f5a61e00640 1 -- start start 2024-08-17T02:45:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.449+0000 7f5a61e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5a5c158ad0 0x7f5a5c0ac070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.449+0000 7f5a61e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5c158ea0 0x7f5a5c0ac5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.449+0000 7f5a61e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5c067d60 con 0x7f5a5c0af410 2024-08-17T02:45:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.449+0000 7f5a61e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5c068600 con 0x7f5a5c158ad0 2024-08-17T02:45:58.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.449+0000 7f5a61e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0aff70 con 0x7f5a5c158ea0 2024-08-17T02:45:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.449+0000 7f5a60a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5a5c158ad0 0x7f5a5c0ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.449+0000 7f5a5be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5c158ea0 0x7f5a5c0ac5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.449+0000 7f5a5be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5c158ea0 0x7f5a5c0ac5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:39404/0 (socket says 172.21.15.16:39404) 2024-08-17T02:45:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.449+0000 7f5a5be00640 1 -- 172.21.15.16:0/4032945138 learned_addr learned my addr 172.21.15.16:0/4032945138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.450+0000 7f5a5be00640 1 -- 172.21.15.16:0/4032945138 >> v2:172.21.15.142:3300/0 conn(0x7f5a5c158ad0 msgr2=0x7f5a5c0ac070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.450+0000 7f5a5be00640 1 --2- 172.21.15.16:0/4032945138 >> v2:172.21.15.142:3300/0 conn(0x7f5a5c158ad0 0x7f5a5c0ac070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.450+0000 7f5a5be00640 1 -- 172.21.15.16:0/4032945138 >> v1:172.21.15.142:6789/0 conn(0x7f5a5c0af410 legacy=0x7f5a5c0af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.450+0000 7f5a5be00640 1 -- 172.21.15.16:0/4032945138 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c0b01d0 con 0x7f5a5c158ea0 2024-08-17T02:45:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.450+0000 7f5a60a00640 1 --2- 172.21.15.16:0/4032945138 >> v2:172.21.15.142:3300/0 conn(0x7f5a5c158ad0 0x7f5a5c0ac070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:45:58.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.451+0000 7f5a5be00640 1 --2- 172.21.15.16:0/4032945138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5c158ea0 0x7f5a5c0ac5b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009910 tx=0x7f5a480091d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40f129282dc6a6c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:58.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.451+0000 7f5a5b400640 1 -- 172.21.15.16:0/4032945138 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4802ad20 con 0x7f5a5c158ea0 2024-08-17T02:45:58.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.452+0000 7f5a5b400640 1 -- 172.21.15.16:0/4032945138 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4802ae80 con 0x7f5a5c158ea0 2024-08-17T02:45:58.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.452+0000 7f5a5b400640 1 -- 172.21.15.16:0/4032945138 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a48032670 con 0x7f5a5c158ea0 2024-08-17T02:45:58.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.452+0000 7f5a61e00640 1 -- 172.21.15.16:0/4032945138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5c158ea0 msgr2=0x7f5a5c0ac5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.452+0000 7f5a61e00640 1 --2- 172.21.15.16:0/4032945138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5c158ea0 0x7f5a5c0ac5b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5a48009910 tx=0x7f5a480091d0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.452+0000 7f5a61e00640 1 -- 172.21.15.16:0/4032945138 shutdown_connections 2024-08-17T02:45:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.452+0000 7f5a61e00640 1 --2- 172.21.15.16:0/4032945138 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5c158ea0 0x7f5a5c0ac5b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.452+0000 7f5a61e00640 1 --2- 172.21.15.16:0/4032945138 >> v2:172.21.15.142:3300/0 conn(0x7f5a5c158ad0 0x7f5a5c0ac070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.452+0000 7f5a61e00640 1 -- 172.21.15.16:0/4032945138 >> 172.21.15.16:0/4032945138 conn(0x7f5a5c096a70 msgr2=0x7f5a5c096e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.452+0000 7f5a61e00640 1 -- 172.21.15.16:0/4032945138 shutdown_connections 2024-08-17T02:45:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.452+0000 7f5a61e00640 1 -- 172.21.15.16:0/4032945138 wait complete. 2024-08-17T02:45:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.454+0000 7f5a61e00640 1 Processor -- start 2024-08-17T02:45:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.454+0000 7f5a61e00640 1 -- start start 2024-08-17T02:45:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.455+0000 7f5a61e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a5c0af410 0x7f5a5c168d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.455+0000 7f5a61e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5c158ad0 0x7f5a5c0cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.455+0000 7f5a61e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5c158ea0 0x7f5a5c0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.455+0000 7f5a61e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0c6c60 con 0x7f5a5c158ea0 2024-08-17T02:45:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.455+0000 7f5a61e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0cde20 con 0x7f5a5c0af410 2024-08-17T02:45:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.455+0000 7f5a61e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c169460 con 0x7f5a5c158ad0 2024-08-17T02:45:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.455+0000 7f5a61400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5c158ea0 0x7f5a5c0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.455+0000 7f5a61400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5c158ea0 0x7f5a5c0b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49442/0 (socket says 172.21.15.16:49442) 2024-08-17T02:45:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.455+0000 7f5a61400640 1 -- 172.21.15.16:0/2274835688 learned_addr learned my addr 172.21.15.16:0/2274835688 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.455+0000 7f5a5be00640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5c158ad0 0x7f5a5c0cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.455+0000 7f5a60a00640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a5c0af410 0x7f5a5c168d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.456+0000 7f5a61400640 1 -- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5c158ad0 msgr2=0x7f5a5c0cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.456+0000 7f5a61400640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5c158ad0 0x7f5a5c0cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.456+0000 7f5a61400640 1 -- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a5c0af410 msgr2=0x7f5a5c168d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.456+0000 7f5a61400640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a5c0af410 0x7f5a5c168d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.456+0000 7f5a61400640 1 -- 172.21.15.16:0/2274835688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a50009390 con 0x7f5a5c158ea0 2024-08-17T02:45:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.456+0000 7f5a61400640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5c158ea0 0x7f5a5c0b2cf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400c7c0 tx=0x7f5a540015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.456+0000 7f5a4fe00640 1 -- 172.21.15.16:0/2274835688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5400a640 con 0x7f5a5c158ea0 2024-08-17T02:45:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.456+0000 7f5a4fe00640 1 -- 172.21.15.16:0/2274835688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5400a7c0 con 0x7f5a5c158ea0 2024-08-17T02:45:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.456+0000 7f5a61e00640 1 -- 172.21.15.16:0/2274835688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a48009450 con 0x7f5a5c158ea0 2024-08-17T02:45:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.456+0000 7f5a4fe00640 1 -- 172.21.15.16:0/2274835688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5400ab80 con 0x7f5a5c158ea0 2024-08-17T02:45:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.456+0000 7f5a61e00640 1 -- 172.21.15.16:0/2274835688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c169a20 con 0x7f5a5c158ea0 2024-08-17T02:45:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.457+0000 7f5a61e00640 1 -- 172.21.15.16:0/2274835688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c169d60 con 0x7f5a5c158ea0 2024-08-17T02:45:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.458+0000 7f5a4fe00640 1 -- 172.21.15.16:0/2274835688 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f5a54003030 con 0x7f5a5c158ea0 2024-08-17T02:45:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.459+0000 7f5a4fe00640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5a3c065150 0x7f5a3c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.459+0000 7f5a4fe00640 1 -- 172.21.15.16:0/2274835688 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5a54082360 con 0x7f5a5c158ea0 2024-08-17T02:45:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.459+0000 7f5a60a00640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5a3c065150 0x7f5a3c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.460+0000 7f5a60a00640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5a3c065150 0x7f5a3c067600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c069e10 tx=0x7f5a500090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.462+0000 7f5a4fe00640 1 -- 172.21.15.16:0/2274835688 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5a54050430 con 0x7f5a5c158ea0 2024-08-17T02:45:58.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.680+0000 7f5a61e00640 1 -- 172.21.15.16:0/2274835688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5a5c071050 con 0x7f5a5c158ea0 2024-08-17T02:45:58.685 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-08-17T02:45:58.681+0000 7f23cec00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-17T02:45:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.686+0000 7f5a4fe00640 1 -- 172.21.15.16:0/2274835688 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5a54053a50 con 0x7f5a5c158ea0 2024-08-17T02:45:58.687 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-17T02:45:58.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.688+0000 7f5a61e00640 1 -- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5a3c065150 msgr2=0x7f5a3c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.688+0000 7f5a61e00640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5a3c065150 0x7f5a3c067600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c069e10 tx=0x7f5a500090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.688+0000 7f5a61e00640 1 -- 172.21.15.16:0/2274835688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5c158ea0 msgr2=0x7f5a5c0b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.688+0000 7f5a61e00640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5c158ea0 0x7f5a5c0b2cf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400c7c0 tx=0x7f5a540015f0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.688+0000 7f5a61e00640 1 -- 172.21.15.16:0/2274835688 shutdown_connections 2024-08-17T02:45:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.688+0000 7f5a61e00640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5a3c065150 0x7f5a3c067600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.688+0000 7f5a61e00640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a5c158ea0 0x7f5a5c0b2cf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.688+0000 7f5a61e00640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a5c158ad0 0x7f5a5c0cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.688+0000 7f5a61e00640 1 --2- 172.21.15.16:0/2274835688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a5c0af410 0x7f5a5c168d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.688+0000 7f5a61e00640 1 -- 172.21.15.16:0/2274835688 >> 172.21.15.16:0/2274835688 conn(0x7f5a5c096a70 msgr2=0x7f5a5c155780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.688+0000 7f5a61e00640 1 -- 172.21.15.16:0/2274835688 shutdown_connections 2024-08-17T02:45:58.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.688+0000 7f5a61e00640 1 -- 172.21.15.16:0/2274835688 wait complete. 2024-08-17T02:45:58.691 INFO:tasks.ceph.mds.c.smithi016.stderr: -1967> 2024-08-17T02:45:24.323+0000 7f5694400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-17T02:45:58.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-08-17T02:45:58.732 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-08-17T02:45:58.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.861+0000 7fc6d0c00640 1 Processor -- start 2024-08-17T02:45:58.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.861+0000 7fc6d0c00640 1 -- start start 2024-08-17T02:45:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.862+0000 7fc6d0c00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6cc155980 0x7fc6cc156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.862+0000 7fc6d0c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc0af580 0x7fc6cc0afa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.862+0000 7fc6d0c00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6cc158400 con 0x7fc6cc0a54d0 2024-08-17T02:45:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.862+0000 7fc6d0c00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6cc0aff40 con 0x7fc6cc155980 2024-08-17T02:45:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.862+0000 7fc6d0c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc1577f0 con 0x7fc6cc0af580 2024-08-17T02:45:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.863+0000 7fc6c9e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6cc155980 0x7fc6cc156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.863+0000 7fc6c9400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc6cc0a54d0 0x7fc6cc155d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56228/0 (socket says 172.21.15.16:56228) 2024-08-17T02:45:58.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.863+0000 7fc6c9400640 1 -- 172.21.15.16:0/646678944 learned_addr learned my addr 172.21.15.16:0/646678944 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.863+0000 7fc6ca800640 1 --2- 172.21.15.16:0/646678944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc0af580 0x7fc6cc0afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.863+0000 7fc6c9e00640 1 -- 172.21.15.16:0/646678944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc0af580 msgr2=0x7fc6cc0afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:45:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.863+0000 7fc6c9e00640 1 --2- 172.21.15.16:0/646678944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc0af580 0x7fc6cc0afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.863+0000 7fc6c9e00640 1 -- 172.21.15.16:0/646678944 >> v1:172.21.15.16:6789/0 conn(0x7fc6cc0a54d0 legacy=0x7fc6cc155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.863+0000 7fc6c9e00640 1 -- 172.21.15.16:0/646678944 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc157ad0 con 0x7fc6cc155980 2024-08-17T02:45:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.863+0000 7fc6c8a00640 1 -- 172.21.15.16:0/646678944 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804809808 0 0) 0x7fc6cc158400 con 0x7fc6cc0a54d0 2024-08-17T02:45:58.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.864+0000 7fc6c9e00640 1 --2- 172.21.15.16:0/646678944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc155980 0x7fc6cc156f70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009850 tx=0x7fc6bc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf738e8979819e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.865+0000 7fc6c8a00640 1 -- 172.21.15.16:0/646678944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc02ad90 con 0x7fc6cc155980 2024-08-17T02:45:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.865+0000 7fc6c8a00640 1 -- 172.21.15.16:0/646678944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc032440 con 0x7fc6cc155980 2024-08-17T02:45:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.865+0000 7fc6c8a00640 1 -- 172.21.15.16:0/646678944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc0327e0 con 0x7fc6cc155980 2024-08-17T02:45:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.865+0000 7fc6d0c00640 1 -- 172.21.15.16:0/646678944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc155980 msgr2=0x7fc6cc156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.865+0000 7fc6d0c00640 1 --2- 172.21.15.16:0/646678944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc155980 0x7fc6cc156f70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009850 tx=0x7fc6bc0090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.865+0000 7fc6d0c00640 1 -- 172.21.15.16:0/646678944 shutdown_connections 2024-08-17T02:45:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.865+0000 7fc6d0c00640 1 --2- 172.21.15.16:0/646678944 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc0af580 0x7fc6cc0afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.865+0000 7fc6d0c00640 1 --2- 172.21.15.16:0/646678944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc155980 0x7fc6cc156f70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.865+0000 7fc6d0c00640 1 -- 172.21.15.16:0/646678944 >> 172.21.15.16:0/646678944 conn(0x7fc6cc067750 msgr2=0x7fc6cc0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:58.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.865+0000 7fc6d0c00640 1 -- 172.21.15.16:0/646678944 shutdown_connections 2024-08-17T02:45:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.865+0000 7fc6d0c00640 1 -- 172.21.15.16:0/646678944 wait complete. 2024-08-17T02:45:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.867+0000 7fc6d0c00640 1 Processor -- start 2024-08-17T02:45:58.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.867+0000 7fc6d0c00640 1 -- start start 2024-08-17T02:45:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6d0c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc0a54d0 0x7fc6cc13ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6d0c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc0af580 0x7fc6cc0accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6d0c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc155980 0x7fc6cc0ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6d0c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc153880 con 0x7fc6cc0af580 2024-08-17T02:45:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6d0c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc152c70 con 0x7fc6cc155980 2024-08-17T02:45:58.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6d0c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc152f50 con 0x7fc6cc0a54d0 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6c9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc0af580 0x7fc6cc0accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6c9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc0af580 0x7fc6cc0accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49492/0 (socket says 172.21.15.16:49492) 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6c9400640 1 -- 172.21.15.16:0/3229351594 learned_addr learned my addr 172.21.15.16:0/3229351594 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6c9e00640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc0a54d0 0x7fc6cc13ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6ca800640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc155980 0x7fc6cc0ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6c9400640 1 -- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc0a54d0 msgr2=0x7fc6cc13ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6c9400640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc0a54d0 0x7fc6cc13ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6c9400640 1 -- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc155980 msgr2=0x7fc6cc0ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6c9400640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc155980 0x7fc6cc0ad1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6c9400640 1 -- 172.21.15.16:0/3229351594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc009390 con 0x7fc6cc0af580 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.868+0000 7fc6ca800640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc155980 0x7fc6cc0ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.869+0000 7fc6c9400640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc0af580 0x7fc6cc0accb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc6b800c510 tx=0x7fc6b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:58.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.869+0000 7fc6c1600640 1 -- 172.21.15.16:0/3229351594 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8005aa0 con 0x7fc6cc0af580 2024-08-17T02:45:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.869+0000 7fc6c1600640 1 -- 172.21.15.16:0/3229351594 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b8005c00 con 0x7fc6cc0af580 2024-08-17T02:45:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.869+0000 7fc6c1600640 1 -- 172.21.15.16:0/3229351594 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b80070d0 con 0x7fc6cc0af580 2024-08-17T02:45:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.869+0000 7fc6d0c00640 1 -- 172.21.15.16:0/3229351594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc153230 con 0x7fc6cc0af580 2024-08-17T02:45:58.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.869+0000 7fc6d0c00640 1 -- 172.21.15.16:0/3229351594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc140440 con 0x7fc6cc0af580 2024-08-17T02:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.870+0000 7fc6d0c00640 1 -- 172.21.15.16:0/3229351594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc153a30 con 0x7fc6cc0af580 2024-08-17T02:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.871+0000 7fc6c1600640 1 -- 172.21.15.16:0/3229351594 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fc6b8007500 con 0x7fc6cc0af580 2024-08-17T02:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.872+0000 7fc6c1600640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc6ac065150 0x7fc6ac067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.872+0000 7fc6c1600640 1 -- 172.21.15.16:0/3229351594 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fc6b8081a70 con 0x7fc6cc0af580 2024-08-17T02:45:58.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.872+0000 7fc6c9e00640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc6ac065150 0x7fc6ac067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:58.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.873+0000 7fc6c9e00640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc6ac065150 0x7fc6ac067600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc034000 tx=0x7fc6bc032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:58.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:58.876+0000 7fc6c1600640 1 -- 172.21.15.16:0/3229351594 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc6b804fa40 con 0x7fc6cc0af580 2024-08-17T02:45:59.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.101+0000 7fc6d0c00640 1 -- 172.21.15.16:0/3229351594 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc6cc071050 con 0x7fc6cc0af580 2024-08-17T02:45:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.686+0000 7fc6c1600640 1 -- 172.21.15.16:0/3229351594 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc6b8053060 con 0x7fc6cc0af580 2024-08-17T02:45:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.689+0000 7fc6d0c00640 1 -- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc6ac065150 msgr2=0x7fc6ac067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.689+0000 7fc6d0c00640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc6ac065150 0x7fc6ac067600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc034000 tx=0x7fc6bc032000 comp rx=0 tx=0).stop 2024-08-17T02:45:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.689+0000 7fc6d0c00640 1 -- 172.21.15.16:0/3229351594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc0af580 msgr2=0x7fc6cc0accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.690+0000 7fc6d0c00640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc0af580 0x7fc6cc0accb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc6b800c510 tx=0x7fc6b800c910 comp rx=0 tx=0).stop 2024-08-17T02:45:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.690+0000 7fc6d0c00640 1 -- 172.21.15.16:0/3229351594 shutdown_connections 2024-08-17T02:45:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.690+0000 7fc6d0c00640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc6ac065150 0x7fc6ac067600 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.690+0000 7fc6d0c00640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6cc155980 0x7fc6cc0ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:59.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.690+0000 7fc6d0c00640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6cc0af580 0x7fc6cc0accb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:59.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.690+0000 7fc6d0c00640 1 --2- 172.21.15.16:0/3229351594 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc6cc0a54d0 0x7fc6cc13ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:59.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.690+0000 7fc6d0c00640 1 -- 172.21.15.16:0/3229351594 >> 172.21.15.16:0/3229351594 conn(0x7fc6cc067750 msgr2=0x7fc6cc154070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:59.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.690+0000 7fc6d0c00640 1 -- 172.21.15.16:0/3229351594 shutdown_connections 2024-08-17T02:45:59.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.690+0000 7fc6d0c00640 1 -- 172.21.15.16:0/3229351594 wait complete. 2024-08-17T02:45:59.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-17T02:45:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.909+0000 7f5a53e00640 1 Processor -- start 2024-08-17T02:45:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.909+0000 7f5a53e00640 1 -- start start 2024-08-17T02:45:59.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.910+0000 7f5a53e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f5a4c06e5f0 0x7f5a4c0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:59.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.910+0000 7f5a53e00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4c0ae9b0 con 0x7f5a4c156c60 2024-08-17T02:45:59.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.910+0000 7f5a53e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4c0ab320 con 0x7f5a4c156890 2024-08-17T02:45:59.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.910+0000 7f5a53e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4c0aa730 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.910+0000 7f5a51e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5a4c156c60 0x7f5a4c0ca3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56230/0 (socket says 172.21.15.16:56230) 2024-08-17T02:45:59.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.910+0000 7f5a51e00640 1 -- 172.21.15.16:0/2845970352 learned_addr learned my addr 172.21.15.16:0/2845970352 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:45:59.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.911+0000 7f5a50a00640 1 --2- 172.21.15.16:0/2845970352 >> v2:172.21.15.142:3300/0 conn(0x7f5a4c06e5f0 0x7f5a4c0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:59.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.911+0000 7f5a4be00640 1 -- 172.21.15.16:0/2845970352 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494415739 0 0) 0x7f5a4c0ae9b0 con 0x7f5a4c156c60 2024-08-17T02:45:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.911+0000 7f5a4be00640 1 -- 172.21.15.16:0/2845970352 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a340034e0 con 0x7f5a4c156c60 2024-08-17T02:45:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.911+0000 7f5a50a00640 1 -- 172.21.15.16:0/2845970352 >> v1:172.21.15.142:6789/0 conn(0x7f5a4c156890 legacy=0x7f5a4c06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.911+0000 7f5a4be00640 1 -- 172.21.15.16:0/2845970352 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298282064 0 0) 0x7f5a4c0ab320 con 0x7f5a4c156890 2024-08-17T02:45:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.911+0000 7f5a50a00640 1 -- 172.21.15.16:0/2845970352 >> v1:172.21.15.16:6789/0 conn(0x7f5a4c156c60 legacy=0x7f5a4c0ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.911+0000 7f5a50a00640 1 -- 172.21.15.16:0/2845970352 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c0aa9f0 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.912+0000 7f5a50a00640 1 --2- 172.21.15.16:0/2845970352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a4c06e5f0 0x7f5a4c0c8130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009850 tx=0x7f5a380090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f126e11b1a71ac4d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:59.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.913+0000 7f5a4be00640 1 -- 172.21.15.16:0/2845970352 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3802ad90 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.913+0000 7f5a4be00640 1 -- 172.21.15.16:0/2845970352 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a38032440 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.913+0000 7f5a4be00640 1 -- 172.21.15.16:0/2845970352 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380327e0 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.913+0000 7f5a53e00640 1 -- 172.21.15.16:0/2845970352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a4c06e5f0 msgr2=0x7f5a4c0c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:59.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.913+0000 7f5a53e00640 1 --2- 172.21.15.16:0/2845970352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a4c06e5f0 0x7f5a4c0c8130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009850 tx=0x7f5a380090f0 comp rx=0 tx=0).stop 2024-08-17T02:45:59.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.913+0000 7f5a53e00640 1 -- 172.21.15.16:0/2845970352 shutdown_connections 2024-08-17T02:45:59.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.913+0000 7f5a53e00640 1 --2- 172.21.15.16:0/2845970352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a4c06e5f0 0x7f5a4c0c8130 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:59.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.913+0000 7f5a53e00640 1 -- 172.21.15.16:0/2845970352 >> 172.21.15.16:0/2845970352 conn(0x7f5a4c067750 msgr2=0x7f5a4c06c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:45:59.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.913+0000 7f5a53e00640 1 -- 172.21.15.16:0/2845970352 shutdown_connections 2024-08-17T02:45:59.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.913+0000 7f5a53e00640 1 -- 172.21.15.16:0/2845970352 wait complete. 2024-08-17T02:45:59.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.915+0000 7f5a53e00640 1 Processor -- start 2024-08-17T02:45:59.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.915+0000 7f5a53e00640 1 -- start start 2024-08-17T02:45:59.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a53e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c06e5f0 0x7f5a4c0a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:59.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a53e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a4c156890 0x7f5a4c069a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:59.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a53e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a4c156c60 0x7f5a4c069fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:59.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a53e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0a7020 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a53e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0a7b60 con 0x7f5a4c156890 2024-08-17T02:45:59.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a53e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0a7230 con 0x7f5a4c156c60 2024-08-17T02:45:59.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a51400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c06e5f0 0x7f5a4c0a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:59.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a51400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c06e5f0 0x7f5a4c0a9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49504/0 (socket says 172.21.15.16:49504) 2024-08-17T02:45:59.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a51400640 1 -- 172.21.15.16:0/3250551852 learned_addr learned my addr 172.21.15.16:0/3250551852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:45:59.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a50a00640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a4c156890 0x7f5a4c069a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:59.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a51e00640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a4c156c60 0x7f5a4c069fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:59.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a51400640 1 -- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a4c156c60 msgr2=0x7f5a4c069fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:59.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a51400640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a4c156c60 0x7f5a4c069fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:59.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a51400640 1 -- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a4c156890 msgr2=0x7f5a4c069a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:45:59.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a51400640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a4c156890 0x7f5a4c069a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:45:59.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a51400640 1 -- 172.21.15.16:0/3250551852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a38009390 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.916+0000 7f5a51e00640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a4c156c60 0x7f5a4c069fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:45:59.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.917+0000 7f5a51400640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c06e5f0 0x7f5a4c0a9d00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000c5b0 tx=0x7f5a40007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:59.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.917+0000 7f5a48c00640 1 -- 172.21.15.16:0/3250551852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40012630 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.917+0000 7f5a48c00640 1 -- 172.21.15.16:0/3250551852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a400127d0 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.917+0000 7f5a48c00640 1 -- 172.21.15.16:0/3250551852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a40012b90 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.917+0000 7f5a53e00640 1 -- 172.21.15.16:0/3250551852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4c0a7510 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.917+0000 7f5a53e00640 1 -- 172.21.15.16:0/3250551852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4c0aa240 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.918+0000 7f5a53e00640 1 -- 172.21.15.16:0/3250551852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4c0710c0 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.919+0000 7f5a48c00640 1 -- 172.21.15.16:0/3250551852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f5a4001b400 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.920+0000 7f5a48c00640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5a2c065150 0x7f5a2c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:45:59.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.920+0000 7f5a48c00640 1 -- 172.21.15.16:0/3250551852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f5a40082190 con 0x7f5a4c06e5f0 2024-08-17T02:45:59.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.920+0000 7f5a50a00640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5a2c065150 0x7f5a2c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:45:59.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.921+0000 7f5a50a00640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5a2c065150 0x7f5a2c067600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5a38034000 tx=0x7f5a38032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:45:59.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:45:59.924+0000 7f5a48c00640 1 -- 172.21.15.16:0/3250551852 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5a40050160 con 0x7f5a4c06e5f0 2024-08-17T02:46:00.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.111+0000 7f5a53e00640 1 -- 172.21.15.16:0/3250551852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5a4c063d20 con 0x7f5a4c06e5f0 2024-08-17T02:46:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.692+0000 7f5a48c00640 1 -- 172.21.15.16:0/3250551852 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5a40053780 con 0x7f5a4c06e5f0 2024-08-17T02:46:00.693 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-08-17T02:46:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.695+0000 7f5a53e00640 1 -- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5a2c065150 msgr2=0x7f5a2c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.695+0000 7f5a53e00640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5a2c065150 0x7f5a2c067600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5a38034000 tx=0x7f5a38032000 comp rx=0 tx=0).stop 2024-08-17T02:46:00.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.695+0000 7f5a53e00640 1 -- 172.21.15.16:0/3250551852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c06e5f0 msgr2=0x7f5a4c0a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.695+0000 7f5a53e00640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c06e5f0 0x7f5a4c0a9d00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000c5b0 tx=0x7f5a40007d60 comp rx=0 tx=0).stop 2024-08-17T02:46:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.695+0000 7f5a53e00640 1 -- 172.21.15.16:0/3250551852 shutdown_connections 2024-08-17T02:46:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.695+0000 7f5a53e00640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5a2c065150 0x7f5a2c067600 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.695+0000 7f5a53e00640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5a4c156c60 0x7f5a4c069fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.695+0000 7f5a53e00640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a4c156890 0x7f5a4c069a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.695+0000 7f5a53e00640 1 --2- 172.21.15.16:0/3250551852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a4c06e5f0 0x7f5a4c0a9d00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.695+0000 7f5a53e00640 1 -- 172.21.15.16:0/3250551852 >> 172.21.15.16:0/3250551852 conn(0x7f5a4c067750 msgr2=0x7f5a4c1530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.695+0000 7f5a53e00640 1 -- 172.21.15.16:0/3250551852 shutdown_connections 2024-08-17T02:46:00.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.695+0000 7f5a53e00640 1 -- 172.21.15.16:0/3250551852 wait complete. 2024-08-17T02:46:00.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-17T02:46:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.871+0000 7f90d5400640 1 Processor -- start 2024-08-17T02:46:00.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.872+0000 7f90d5400640 1 -- start start 2024-08-17T02:46:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.873+0000 7f90d5400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90d0155880 0x7f90d00ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.873+0000 7f90d5400640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d0157840 con 0x7f90d00aeb70 2024-08-17T02:46:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.873+0000 7f90d5400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d0156c30 con 0x7f90d00af800 2024-08-17T02:46:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.873+0000 7f90d5400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d0156f10 con 0x7f90d0155880 2024-08-17T02:46:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.873+0000 7f90cfe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90d0155880 0x7f90d00ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.873+0000 7f90cfe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90d0155880 0x7f90d00ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49506/0 (socket says 172.21.15.16:49506) 2024-08-17T02:46:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.873+0000 7f90cfe00640 1 -- 172.21.15.16:0/2502649600 learned_addr learned my addr 172.21.15.16:0/2502649600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.874+0000 7f90cfe00640 1 -- 172.21.15.16:0/2502649600 >> v1:172.21.15.142:6789/0 conn(0x7f90d00af800 legacy=0x7f90d0157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:00.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.874+0000 7f90cfe00640 1 -- 172.21.15.16:0/2502649600 >> v1:172.21.15.16:6789/0 conn(0x7f90d00aeb70 legacy=0x7f90d00af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.874+0000 7f90cfe00640 1 -- 172.21.15.16:0/2502649600 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d01571f0 con 0x7f90d0155880 2024-08-17T02:46:00.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.874+0000 7f90cf400640 1 -- 172.21.15.16:0/2502649600 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176087396 0 0) 0x7f90d0157840 con 0x7f90d00aeb70 2024-08-17T02:46:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.875+0000 7f90cfe00640 1 --2- 172.21.15.16:0/2502649600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d0155880 0x7f90d00ae630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009910 tx=0x7f90c00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f8999a9a793f08 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.875+0000 7f90cf400640 1 -- 172.21.15.16:0/2502649600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c002ae60 con 0x7f90d0155880 2024-08-17T02:46:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.875+0000 7f90cf400640 1 -- 172.21.15.16:0/2502649600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c0032440 con 0x7f90d0155880 2024-08-17T02:46:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.875+0000 7f90cf400640 1 -- 172.21.15.16:0/2502649600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c00327e0 con 0x7f90d0155880 2024-08-17T02:46:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.875+0000 7f90d5400640 1 -- 172.21.15.16:0/2502649600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d0155880 msgr2=0x7f90d00ae630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.875+0000 7f90d5400640 1 --2- 172.21.15.16:0/2502649600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d0155880 0x7f90d00ae630 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009910 tx=0x7f90c00091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.876+0000 7f90d5400640 1 -- 172.21.15.16:0/2502649600 shutdown_connections 2024-08-17T02:46:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.876+0000 7f90d5400640 1 --2- 172.21.15.16:0/2502649600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d0155880 0x7f90d00ae630 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.876+0000 7f90d5400640 1 -- 172.21.15.16:0/2502649600 >> 172.21.15.16:0/2502649600 conn(0x7f90d0067750 msgr2=0x7f90d00cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.876+0000 7f90d5400640 1 -- 172.21.15.16:0/2502649600 shutdown_connections 2024-08-17T02:46:00.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.876+0000 7f90d5400640 1 -- 172.21.15.16:0/2502649600 wait complete. 2024-08-17T02:46:00.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.877+0000 7f90d5400640 1 Processor -- start 2024-08-17T02:46:00.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.877+0000 7f90d5400640 1 -- start start 2024-08-17T02:46:00.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.878+0000 7f90d5400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d00aeb70 0x7f90d01406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:00.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.878+0000 7f90d5400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d00af800 0x7f90d013ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.878+0000 7f90d5400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d0155880 0x7f90d016c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.878+0000 7f90d5400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d00abdf0 con 0x7f90d00af800 2024-08-17T02:46:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.878+0000 7f90d5400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d00ab1e0 con 0x7f90d00aeb70 2024-08-17T02:46:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.878+0000 7f90d5400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d00ab4c0 con 0x7f90d0155880 2024-08-17T02:46:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.878+0000 7f90c7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d00af800 0x7f90d013ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.878+0000 7f90c7400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d00af800 0x7f90d013ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49510/0 (socket says 172.21.15.16:49510) 2024-08-17T02:46:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.878+0000 7f90c7400640 1 -- 172.21.15.16:0/2324554550 learned_addr learned my addr 172.21.15.16:0/2324554550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90cfe00640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d00aeb70 0x7f90d01406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90d4a00640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d0155880 0x7f90d016c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90c7400640 1 -- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d0155880 msgr2=0x7f90d016c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90c7400640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d0155880 0x7f90d016c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:00.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90c7400640 1 -- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d00aeb70 msgr2=0x7f90d01406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90c7400640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d00aeb70 0x7f90d01406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90c7400640 1 -- 172.21.15.16:0/2324554550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c0009450 con 0x7f90d00af800 2024-08-17T02:46:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90cfe00640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d00aeb70 0x7f90d01406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90c7400640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d00af800 0x7f90d013ffa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f90b800c5b0 tx=0x7f90b8007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90c7e00640 1 -- 172.21.15.16:0/2324554550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8012630 con 0x7f90d00af800 2024-08-17T02:46:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90c7e00640 1 -- 172.21.15.16:0/2324554550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b80127d0 con 0x7f90d00af800 2024-08-17T02:46:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90d5400640 1 -- 172.21.15.16:0/2324554550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d00ab7a0 con 0x7f90d00af800 2024-08-17T02:46:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.879+0000 7f90c7e00640 1 -- 172.21.15.16:0/2324554550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b8012b90 con 0x7f90d00af800 2024-08-17T02:46:00.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.880+0000 7f90d5400640 1 -- 172.21.15.16:0/2324554550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d00aa860 con 0x7f90d00af800 2024-08-17T02:46:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.881+0000 7f90d5400640 1 -- 172.21.15.16:0/2324554550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d00aaa80 con 0x7f90d00af800 2024-08-17T02:46:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.881+0000 7f90c7e00640 1 -- 172.21.15.16:0/2324554550 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f90b801b400 con 0x7f90d00af800 2024-08-17T02:46:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.882+0000 7f90c7e00640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f90ac065150 0x7f90ac067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:00.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.882+0000 7f90c7e00640 1 -- 172.21.15.16:0/2324554550 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f90b8081f50 con 0x7f90d00af800 2024-08-17T02:46:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.883+0000 7f90cfe00640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f90ac065150 0x7f90ac067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:00.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.883+0000 7f90cfe00640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f90ac065150 0x7f90ac067600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f90c0034000 tx=0x7f90c0032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:00.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:00.886+0000 7f90c7e00640 1 -- 172.21.15.16:0/2324554550 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f90d00aaa80 con 0x7f90d00af800 2024-08-17T02:46:01.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.070+0000 7f90d5400640 1 -- 172.21.15.16:0/2324554550 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f90d00aaa80 con 0x7f90d00af800 2024-08-17T02:46:01.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.698+0000 7f90c7e00640 1 -- 172.21.15.16:0/2324554550 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f90d00aaa80 con 0x7f90d00af800 2024-08-17T02:46:01.699 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-08-17T02:46:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.702+0000 7f90d5400640 1 -- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f90ac065150 msgr2=0x7f90ac067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.702+0000 7f90d5400640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f90ac065150 0x7f90ac067600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f90c0034000 tx=0x7f90c0032000 comp rx=0 tx=0).stop 2024-08-17T02:46:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.702+0000 7f90d5400640 1 -- 172.21.15.16:0/2324554550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d00af800 msgr2=0x7f90d013ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:01.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.702+0000 7f90d5400640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d00af800 0x7f90d013ffa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f90b800c5b0 tx=0x7f90b8007d60 comp rx=0 tx=0).stop 2024-08-17T02:46:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.703+0000 7f90d5400640 1 -- 172.21.15.16:0/2324554550 shutdown_connections 2024-08-17T02:46:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.703+0000 7f90d5400640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f90ac065150 0x7f90ac067600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.703+0000 7f90d5400640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f90d0155880 0x7f90d016c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.703+0000 7f90d5400640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90d00af800 0x7f90d013ffa0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.703+0000 7f90d5400640 1 --2- 172.21.15.16:0/2324554550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90d00aeb70 0x7f90d01406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.703+0000 7f90d5400640 1 -- 172.21.15.16:0/2324554550 >> 172.21.15.16:0/2324554550 conn(0x7f90d0067750 msgr2=0x7f90d0158f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:01.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.703+0000 7f90d5400640 1 -- 172.21.15.16:0/2324554550 shutdown_connections 2024-08-17T02:46:01.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:01.703+0000 7f90d5400640 1 -- 172.21.15.16:0/2324554550 wait complete. 2024-08-17T02:46:01.722 INFO:tasks.ceph.mds.a:Restarting daemon 2024-08-17T02:46:01.722 INFO:tasks.ceph.mds.a:Stopping old one... 2024-08-17T02:46:01.722 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-08-17T02:46:01.722 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:46:01.723 INFO:tasks.ceph.mds.c:Restarting daemon 2024-08-17T02:46:01.723 INFO:tasks.ceph.mds.c:Stopping old one... 2024-08-17T02:46:01.723 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-08-17T02:46:01.723 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:46:01.723 INFO:tasks.ceph.mds.b:Restarting daemon 2024-08-17T02:46:01.723 INFO:tasks.ceph.mds.b:Stopping old one... 2024-08-17T02:46:01.724 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-08-17T02:46:01.724 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:46:01.724 INFO:tasks.ceph.mds.d:Restarting daemon 2024-08-17T02:46:01.724 INFO:tasks.ceph.mds.d:Stopping old one... 2024-08-17T02:46:01.724 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-17T02:46:01.724 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:46:01.740 INFO:tasks.ceph.mds.c:Stopped 2024-08-17T02:46:01.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-08-17T02:46:01.742 INFO:tasks.ceph.mds.a:Stopped 2024-08-17T02:46:01.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-08-17T02:46:01.743 INFO:tasks.ceph.mds.b:Stopped 2024-08-17T02:46:01.743 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-08-17T02:46:01.787 INFO:tasks.ceph.mds.c:Started 2024-08-17T02:46:01.787 DEBUG:teuthology.parallel:result is None 2024-08-17T02:46:01.789 INFO:tasks.ceph.mds.b:Started 2024-08-17T02:46:01.789 DEBUG:teuthology.parallel:result is None 2024-08-17T02:46:01.791 INFO:tasks.ceph.mds.a:Started 2024-08-17T02:46:01.791 DEBUG:teuthology.parallel:result is None 2024-08-17T02:46:01.918 INFO:tasks.ceph.mds.b.smithi142.stdout:starting mds.b at 2024-08-17T02:46:01.930 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-08-17T02:46:01.931 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-08-17T02:46:06.522 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-08-17T02:46:07.743 INFO:tasks.ceph.mds.d:Stopped 2024-08-17T02:46:07.743 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-08-17T02:46:07.748 INFO:tasks.ceph.mds.d:Started 2024-08-17T02:46:07.748 DEBUG:teuthology.parallel:result is None 2024-08-17T02:46:07.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-08-17T02:46:07.875 INFO:tasks.ceph.mds.d.smithi142.stdout:starting mds.d at 2024-08-17T02:46:07.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.913+0000 7fa4d3a00640 1 Processor -- start 2024-08-17T02:46:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.913+0000 7fa4d3a00640 1 -- start start 2024-08-17T02:46:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.914+0000 7fa4d3a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4cc155a20 0x7fa4cc0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d3a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa4cc06df00 0x7fa4cc0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d3a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4cc156670 con 0x7fa4cc155df0 2024-08-17T02:46:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d3a00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4cc0b0350 con 0x7fa4cc155a20 2024-08-17T02:46:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d3a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4cc06e5e0 con 0x7fa4cc06df00 2024-08-17T02:46:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d1000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4cc155a20 0x7fa4cc0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d1000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4cc155a20 0x7fa4cc0adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48414/0 (socket says 172.21.15.16:48414) 2024-08-17T02:46:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d1000640 1 -- 172.21.15.16:0/4196781833 learned_addr learned my addr 172.21.15.16:0/4196781833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d1a00640 1 --2- 172.21.15.16:0/4196781833 >> v2:172.21.15.142:3300/0 conn(0x7fa4cc06df00 0x7fa4cc0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d1000640 1 -- 172.21.15.16:0/4196781833 >> v2:172.21.15.142:3300/0 conn(0x7fa4cc06df00 msgr2=0x7fa4cc0ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d1000640 1 --2- 172.21.15.16:0/4196781833 >> v2:172.21.15.142:3300/0 conn(0x7fa4cc06df00 0x7fa4cc0ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:07.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d1000640 1 -- 172.21.15.16:0/4196781833 >> v1:172.21.15.142:6789/0 conn(0x7fa4cc155df0 legacy=0x7fa4cc06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:07.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d1000640 1 -- 172.21.15.16:0/4196781833 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc06e8c0 con 0x7fa4cc155a20 2024-08-17T02:46:07.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.915+0000 7fa4d1a00640 1 --2- 172.21.15.16:0/4196781833 >> v2:172.21.15.142:3300/0 conn(0x7fa4cc06df00 0x7fa4cc0ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.916+0000 7fa4d1000640 1 --2- 172.21.15.16:0/4196781833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4cc155a20 0x7fa4cc0adf10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc1565e0 tx=0x7fa4c00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe76be10519a401 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.917+0000 7fa4cb400640 1 -- 172.21.15.16:0/4196781833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c002add0 con 0x7fa4cc155a20 2024-08-17T02:46:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.917+0000 7fa4cb400640 1 -- 172.21.15.16:0/4196781833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c0032440 con 0x7fa4cc155a20 2024-08-17T02:46:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.917+0000 7fa4cb400640 1 -- 172.21.15.16:0/4196781833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0032820 con 0x7fa4cc155a20 2024-08-17T02:46:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.917+0000 7fa4d3a00640 1 -- 172.21.15.16:0/4196781833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4cc155a20 msgr2=0x7fa4cc0adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.917+0000 7fa4d3a00640 1 --2- 172.21.15.16:0/4196781833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4cc155a20 0x7fa4cc0adf10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc1565e0 tx=0x7fa4c00091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.917+0000 7fa4d3a00640 1 -- 172.21.15.16:0/4196781833 shutdown_connections 2024-08-17T02:46:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.917+0000 7fa4d3a00640 1 --2- 172.21.15.16:0/4196781833 >> v2:172.21.15.142:3300/0 conn(0x7fa4cc06df00 0x7fa4cc0ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.917+0000 7fa4d3a00640 1 --2- 172.21.15.16:0/4196781833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4cc155a20 0x7fa4cc0adf10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.917+0000 7fa4d3a00640 1 -- 172.21.15.16:0/4196781833 >> 172.21.15.16:0/4196781833 conn(0x7fa4cc0cb9b0 msgr2=0x7fa4cc0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.917+0000 7fa4d3a00640 1 -- 172.21.15.16:0/4196781833 shutdown_connections 2024-08-17T02:46:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.917+0000 7fa4d3a00640 1 -- 172.21.15.16:0/4196781833 wait complete. 2024-08-17T02:46:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.919+0000 7fa4d3a00640 1 Processor -- start 2024-08-17T02:46:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.919+0000 7fa4d3a00640 1 -- start start 2024-08-17T02:46:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d3a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4cc06df00 0x7fa4cc1483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d3a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4cc155a20 0x7fa4cc15d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d3a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4cc155df0 0x7fa4cc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d3a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc14c460 con 0x7fa4cc155df0 2024-08-17T02:46:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d3a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc14c740 con 0x7fa4cc06df00 2024-08-17T02:46:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d3a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc14ca20 con 0x7fa4cc155a20 2024-08-17T02:46:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4cc155df0 0x7fa4cc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4cc155df0 0x7fa4cc15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48422/0 (socket says 172.21.15.16:48422) 2024-08-17T02:46:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d1a00640 1 -- 172.21.15.16:0/2279305977 learned_addr learned my addr 172.21.15.16:0/2279305977 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d1000640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4cc06df00 0x7fa4cc1483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4cbe00640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4cc155a20 0x7fa4cc15d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d1a00640 1 -- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4cc155a20 msgr2=0x7fa4cc15d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d1a00640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4cc155a20 0x7fa4cc15d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d1a00640 1 -- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4cc06df00 msgr2=0x7fa4cc1483e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d1a00640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4cc06df00 0x7fa4cc1483e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.920+0000 7fa4d1a00640 1 -- 172.21.15.16:0/2279305977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c0009450 con 0x7fa4cc155df0 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.921+0000 7fa4cbe00640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4cc155a20 0x7fa4cc15d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.921+0000 7fa4d1a00640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4cc155df0 0x7fa4cc15dc90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400d150 tx=0x7fa4c4009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.921+0000 7fa4bfe00640 1 -- 172.21.15.16:0/2279305977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4015440 con 0x7fa4cc155df0 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.921+0000 7fa4bfe00640 1 -- 172.21.15.16:0/2279305977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c400f040 con 0x7fa4cc155df0 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.921+0000 7fa4d3a00640 1 -- 172.21.15.16:0/2279305977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4cc14cd00 con 0x7fa4cc155df0 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.921+0000 7fa4bfe00640 1 -- 172.21.15.16:0/2279305977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4015810 con 0x7fa4cc155df0 2024-08-17T02:46:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.921+0000 7fa4d3a00640 1 -- 172.21.15.16:0/2279305977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4cc14d500 con 0x7fa4cc155df0 2024-08-17T02:46:07.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.922+0000 7fa4d3a00640 1 -- 172.21.15.16:0/2279305977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4cc14d200 con 0x7fa4cc155df0 2024-08-17T02:46:07.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.923+0000 7fa4bfe00640 1 -- 172.21.15.16:0/2279305977 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fa4c401e3b0 con 0x7fa4cc155df0 2024-08-17T02:46:07.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.924+0000 7fa4bfe00640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa4ac065150 0x7fa4ac067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.924+0000 7fa4bfe00640 1 -- 172.21.15.16:0/2279305977 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa4c4082c30 con 0x7fa4cc155df0 2024-08-17T02:46:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.924+0000 7fa4d1000640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa4ac065150 0x7fa4ac067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:07.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.925+0000 7fa4d1000640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa4ac065150 0x7fa4ac067600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0034000 tx=0x7fa4c0032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:07.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:07.928+0000 7fa4bfe00640 1 -- 172.21.15.16:0/2279305977 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa4c4050f50 con 0x7fa4cc155df0 2024-08-17T02:46:08.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:08.124+0000 7fa4d3a00640 1 -- 172.21.15.16:0/2279305977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa4cc064db0 con 0x7fa4cc155df0 2024-08-17T02:46:09.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.731+0000 7fa4bfe00640 1 -- 172.21.15.16:0/2279305977 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa4c4054570 con 0x7fa4cc155df0 2024-08-17T02:46:09.731 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2024-08-17T02:46:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.734+0000 7fa4d3a00640 1 -- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa4ac065150 msgr2=0x7fa4ac067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.734+0000 7fa4d3a00640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa4ac065150 0x7fa4ac067600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0034000 tx=0x7fa4c0032000 comp rx=0 tx=0).stop 2024-08-17T02:46:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.734+0000 7fa4d3a00640 1 -- 172.21.15.16:0/2279305977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4cc155df0 msgr2=0x7fa4cc15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:09.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.734+0000 7fa4d3a00640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4cc155df0 0x7fa4cc15dc90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400d150 tx=0x7fa4c4009a50 comp rx=0 tx=0).stop 2024-08-17T02:46:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.734+0000 7fa4d3a00640 1 -- 172.21.15.16:0/2279305977 shutdown_connections 2024-08-17T02:46:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.734+0000 7fa4d3a00640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa4ac065150 0x7fa4ac067600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.734+0000 7fa4d3a00640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4cc155df0 0x7fa4cc15dc90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.734+0000 7fa4d3a00640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa4cc155a20 0x7fa4cc15d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.734+0000 7fa4d3a00640 1 --2- 172.21.15.16:0/2279305977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa4cc06df00 0x7fa4cc1483e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.734+0000 7fa4d3a00640 1 -- 172.21.15.16:0/2279305977 >> 172.21.15.16:0/2279305977 conn(0x7fa4cc0cb9b0 msgr2=0x7fa4cc1578f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.734+0000 7fa4d3a00640 1 -- 172.21.15.16:0/2279305977 shutdown_connections 2024-08-17T02:46:09.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.734+0000 7fa4d3a00640 1 -- 172.21.15.16:0/2279305977 wait complete. 2024-08-17T02:46:09.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-08-17T02:46:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.908+0000 7f67ab200640 1 Processor -- start 2024-08-17T02:46:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.908+0000 7f67ab200640 1 -- start start 2024-08-17T02:46:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.910+0000 7f67ab200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67ac06d7a0 0x7f67ac06db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.910+0000 7f67ab200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ac15ccc0 0x7f67ac15d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.910+0000 7f67ab200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ac0ae250 con 0x7f67ac0acf10 2024-08-17T02:46:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.910+0000 7f67ab200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ac0ad640 con 0x7f67ac06d7a0 2024-08-17T02:46:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.910+0000 7f67ab200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac0ad920 con 0x7f67ac15ccc0 2024-08-17T02:46:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.910+0000 7f67a9e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67ac06d7a0 0x7f67ac06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.910+0000 7f67a9e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67ac06d7a0 0x7f67ac06db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48428/0 (socket says 172.21.15.16:48428) 2024-08-17T02:46:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.910+0000 7f67a9e00640 1 -- 172.21.15.16:0/3375654822 learned_addr learned my addr 172.21.15.16:0/3375654822 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.910+0000 7f67aa800640 1 --2- 172.21.15.16:0/3375654822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ac15ccc0 0x7f67ac15d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.910+0000 7f67a9e00640 1 -- 172.21.15.16:0/3375654822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ac15ccc0 msgr2=0x7f67ac15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.910+0000 7f67a9e00640 1 --2- 172.21.15.16:0/3375654822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ac15ccc0 0x7f67ac15d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:09.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.911+0000 7f67a9e00640 1 -- 172.21.15.16:0/3375654822 >> v1:172.21.15.142:6789/0 conn(0x7f67ac0acf10 legacy=0x7f67ac06e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.911+0000 7f67a9e00640 1 -- 172.21.15.16:0/3375654822 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac0adc00 con 0x7f67ac06d7a0 2024-08-17T02:46:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.912+0000 7f67a9e00640 1 --2- 172.21.15.16:0/3375654822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ac06d7a0 0x7f67ac06db70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f679c009910 tx=0x7f679c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3752cfd05e487fd1 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.912+0000 7f67a8a00640 1 -- 172.21.15.16:0/3375654822 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c02ae40 con 0x7f67ac06d7a0 2024-08-17T02:46:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.912+0000 7f67a8a00640 1 -- 172.21.15.16:0/3375654822 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c032440 con 0x7f67ac06d7a0 2024-08-17T02:46:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.912+0000 7f67a8a00640 1 -- 172.21.15.16:0/3375654822 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c032820 con 0x7f67ac06d7a0 2024-08-17T02:46:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.912+0000 7f67ab200640 1 -- 172.21.15.16:0/3375654822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ac06d7a0 msgr2=0x7f67ac06db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.912+0000 7f67ab200640 1 --2- 172.21.15.16:0/3375654822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ac06d7a0 0x7f67ac06db70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f679c009910 tx=0x7f679c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.912+0000 7f67ab200640 1 -- 172.21.15.16:0/3375654822 shutdown_connections 2024-08-17T02:46:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.912+0000 7f67ab200640 1 --2- 172.21.15.16:0/3375654822 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ac15ccc0 0x7f67ac15d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.912+0000 7f67ab200640 1 --2- 172.21.15.16:0/3375654822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ac06d7a0 0x7f67ac06db70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.912+0000 7f67ab200640 1 -- 172.21.15.16:0/3375654822 >> 172.21.15.16:0/3375654822 conn(0x7f67ac067750 msgr2=0x7f67ac0cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.913+0000 7f67ab200640 1 -- 172.21.15.16:0/3375654822 shutdown_connections 2024-08-17T02:46:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.913+0000 7f67ab200640 1 -- 172.21.15.16:0/3375654822 wait complete. 2024-08-17T02:46:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.914+0000 7f67ab200640 1 Processor -- start 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.914+0000 7f67ab200640 1 -- start start 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.915+0000 7f67ab200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ac06d7a0 0x7f67ac158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.915+0000 7f67ab200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ac0acf10 0x7f67ac14e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.915+0000 7f67ab200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ac15ccc0 0x7f67ac15a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.915+0000 7f67ab200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac152630 con 0x7f67ac0acf10 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.915+0000 7f67ab200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac159060 con 0x7f67ac06d7a0 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.915+0000 7f67ab200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac159340 con 0x7f67ac15ccc0 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.915+0000 7f67a9e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ac06d7a0 0x7f67ac158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.915+0000 7f67a9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ac0acf10 0x7f67ac14e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.915+0000 7f67a9e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ac06d7a0 0x7f67ac158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:57880/0 (socket says 172.21.15.16:57880) 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.915+0000 7f67a9e00640 1 -- 172.21.15.16:0/2377887788 learned_addr learned my addr 172.21.15.16:0/2377887788 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.915+0000 7f67aa800640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ac15ccc0 0x7f67ac15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.916+0000 7f67a9400640 1 -- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ac15ccc0 msgr2=0x7f67ac15a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.916+0000 7f67a9400640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ac15ccc0 0x7f67ac15a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.916+0000 7f67a9400640 1 -- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ac06d7a0 msgr2=0x7f67ac158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.916+0000 7f67a9400640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ac06d7a0 0x7f67ac158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.916+0000 7f67a9400640 1 -- 172.21.15.16:0/2377887788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c009450 con 0x7f67ac0acf10 2024-08-17T02:46:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.916+0000 7f67a9e00640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ac06d7a0 0x7f67ac158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:46:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.916+0000 7f67a9400640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ac0acf10 0x7f67ac14e920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f679800c410 tx=0x7f6798007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.916+0000 7f67a1600640 1 -- 172.21.15.16:0/2377887788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798011590 con 0x7f67ac0acf10 2024-08-17T02:46:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.916+0000 7f67a1600640 1 -- 172.21.15.16:0/2377887788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67980116f0 con 0x7f67ac0acf10 2024-08-17T02:46:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.916+0000 7f67a1600640 1 -- 172.21.15.16:0/2377887788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798011ab0 con 0x7f67ac0acf10 2024-08-17T02:46:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.916+0000 7f67ab200640 1 -- 172.21.15.16:0/2377887788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ac159620 con 0x7f67ac0acf10 2024-08-17T02:46:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.917+0000 7f67ab200640 1 -- 172.21.15.16:0/2377887788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ac159be0 con 0x7f67ac0acf10 2024-08-17T02:46:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.918+0000 7f67ab200640 1 -- 172.21.15.16:0/2377887788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ac159e30 con 0x7f67ac0acf10 2024-08-17T02:46:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.918+0000 7f67a1600640 1 -- 172.21.15.16:0/2377887788 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f679801b3b0 con 0x7f67ac0acf10 2024-08-17T02:46:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.920+0000 7f67a1600640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f678c065150 0x7f678c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.920+0000 7f67a1600640 1 -- 172.21.15.16:0/2377887788 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6798081560 con 0x7f67ac0acf10 2024-08-17T02:46:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.920+0000 7f67a9e00640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f678c065150 0x7f678c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.921+0000 7f67a9e00640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f678c065150 0x7f678c067600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f679c009910 tx=0x7f679c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:09.923+0000 7f67a1600640 1 -- 172.21.15.16:0/2377887788 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f679804f980 con 0x7f67ac0acf10 2024-08-17T02:46:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.172+0000 7f67ab200640 1 -- 172.21.15.16:0/2377887788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f67ac0ab980 con 0x7f67ac0acf10 2024-08-17T02:46:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.173+0000 7f67a1600640 1 -- 172.21.15.16:0/2377887788 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f67ac0ab980 con 0x7f67ac0acf10 2024-08-17T02:46:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:46:10.173 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-17T02:46:10.173 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2024-08-17T02:46:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2024-08-17T02:46:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGDsBmqW8wMxAAq+fTCp8vlCJN6+cR/DX7EA==", 2024-08-17T02:46:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-08-17T02:46:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-08-17T02:46:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-08-17T02:46:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.174 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHDsBmY02mChAAwbxC+Xnlh+vHDhbrKl0MHg==", 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHDsBmMhIjARAAmNH2mDnE7u+gPFZBvH0YOA==", 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-08-17T02:46:10.175 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHDsBmQav9ExAAlAXMmqzLG2nF2c8Jo4pMzg==", 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.176 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2024-08-17T02:46:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCJDsBmDc6YBRAAr1Dv8OP3NemJBuGlEKic+w==", 2024-08-17T02:46:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-08-17T02:46:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-08-17T02:46:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.177 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCMDsBmhIWNCRAAZazsazPPVNJkGx16uvsmhg==", 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2024-08-17T02:46:10.178 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCPDsBm7tEHDRAAfoe57cHfMYzOyluVRl7OWg==", 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCSDsBmhBH0EBAAy2nZuWtACHGHNBXHSs0x5Q==", 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.179 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCYDsBmUzYaCBAAD1a9iSf7v8jH3YImMPUX2A==", 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCbDsBmQjYkDBAAdlCM8QfcMLK5dBRR1LMIIQ==", 2024-08-17T02:46:10.180 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-08-17T02:46:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-08-17T02:46:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2024-08-17T02:46:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCeDsBmsLwEEhAAMIvs0maEzEEHCxBwO0Ae/A==", 2024-08-17T02:46:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-08-17T02:46:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-08-17T02:46:10.181 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2024-08-17T02:46:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQChDsBmuGZmFRAAIho6JxP3NTmgOsuCmH3x0w==", 2024-08-17T02:46:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2024-08-17T02:46:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2024-08-17T02:46:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.182 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHDsBmLsQ9HBAA8Lj9LuvNWPNmRDuhWQDMpQ==", 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHDsBmAJZaKBAALuHMnu/+CvfwswA+EC/zjg==", 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.183 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHDsBmNQ9TIhAACLZVzo0XXehLBoawb5yqzw==", 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-08-17T02:46:10.184 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCHDsBmWZpeLhAAozMeV17bYm3xQMnkbof5vQ==", 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.185 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGDsBm3Wk5BRAA2+cAkEULWAK3uklaYogFtg==", 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mds", 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCtDsBmrtN2KRAAoRcR5IYbdosuZog4ECHKqg==", 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.186 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mds" 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mgr", 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCtDsBm99x2KRAAPaforTAsA0S6kVruKWIC3Q==", 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mgr" 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-osd", 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCtDsBmcel2KRAA+0QGYuEilKyrVJZhMbXZVA==", 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.187 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-osd" 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd", 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCtDsBm5PF2KRAAiDiyEj2qr3y5LQ+4kVyd3w==", 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd" 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCtDsBmq/p2KRAAoZKwnhu5gC21eBTKIbHuNw==", 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-08-17T02:46:10.188 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rgw", 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCtDsBmBgR3KRAAW/erRt0M6EMXBoDY2oGwag==", 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rgw" 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGDsBm3fn6LBAAMMskmYZc6ASOkn+stKKllw==", 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.189 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQCGDsBm5S2qJhAAbhnTqOJjI5AwT0ET6q/GRg==", 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:10.190 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-08-17T02:46:10.191 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-17T02:46:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.176+0000 7f67ab200640 1 -- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f678c065150 msgr2=0x7f678c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.176+0000 7f67ab200640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f678c065150 0x7f678c067600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f679c009910 tx=0x7f679c032000 comp rx=0 tx=0).stop 2024-08-17T02:46:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.176+0000 7f67ab200640 1 -- 172.21.15.16:0/2377887788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ac0acf10 msgr2=0x7f67ac14e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.176+0000 7f67ab200640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ac0acf10 0x7f67ac14e920 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f679800c410 tx=0x7f6798007e30 comp rx=0 tx=0).stop 2024-08-17T02:46:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.176+0000 7f67ab200640 1 -- 172.21.15.16:0/2377887788 shutdown_connections 2024-08-17T02:46:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.176+0000 7f67ab200640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f678c065150 0x7f678c067600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.176+0000 7f67ab200640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f67ac15ccc0 0x7f67ac15a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.176+0000 7f67ab200640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ac0acf10 0x7f67ac14e920 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.176+0000 7f67ab200640 1 --2- 172.21.15.16:0/2377887788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67ac06d7a0 0x7f67ac158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.176+0000 7f67ab200640 1 -- 172.21.15.16:0/2377887788 >> 172.21.15.16:0/2377887788 conn(0x7f67ac067750 msgr2=0x7f67ac15bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.176+0000 7f67ab200640 1 -- 172.21.15.16:0/2377887788 shutdown_connections 2024-08-17T02:46:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.176+0000 7f67ab200640 1 -- 172.21.15.16:0/2377887788 wait complete. 2024-08-17T02:46:10.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-08-17T02:46:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.345+0000 7fbd8c800640 1 Processor -- start 2024-08-17T02:46:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.345+0000 7fbd8c800640 1 -- start start 2024-08-17T02:46:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.346+0000 7fbd8c800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd84155a20 0x7fbd840adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.346+0000 7fbd8c800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbd84155df0 0x7fbd840ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.346+0000 7fbd8c800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd84156670 con 0x7fbd8406d7f0 2024-08-17T02:46:10.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.346+0000 7fbd8c800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd840b0350 con 0x7fbd84155df0 2024-08-17T02:46:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.346+0000 7fbd8c800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8406e5e0 con 0x7fbd84155a20 2024-08-17T02:46:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.347+0000 7fbd89400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbd84155df0 0x7fbd840ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.347+0000 7fbd89400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fbd84155df0 0x7fbd840ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:57894/0 (socket says 172.21.15.16:57894) 2024-08-17T02:46:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.347+0000 7fbd89400640 1 -- 172.21.15.16:0/1909139221 learned_addr learned my addr 172.21.15.16:0/1909139221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.347+0000 7fbd89e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd84155a20 0x7fbd840adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.347+0000 7fbd8a800640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fbd8406d7f0 0x7fbd840ac790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:44266/0 (socket says 172.21.15.16:44266) 2024-08-17T02:46:10.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.347+0000 7fbd89400640 1 -- 172.21.15.16:0/1909139221 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd84155a20 msgr2=0x7fbd840adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:10.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.347+0000 7fbd89400640 1 --2- 172.21.15.16:0/1909139221 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd84155a20 0x7fbd840adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.347+0000 7fbd89400640 1 -- 172.21.15.16:0/1909139221 >> v1:172.21.15.142:6789/0 conn(0x7fbd8406d7f0 legacy=0x7fbd840ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:10.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.347+0000 7fbd89400640 1 -- 172.21.15.16:0/1909139221 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8406e8c0 con 0x7fbd84155df0 2024-08-17T02:46:10.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.347+0000 7fbd88a00640 1 -- 172.21.15.16:0/1909139221 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098766631 0 0) 0x7fbd84156670 con 0x7fbd8406d7f0 2024-08-17T02:46:10.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.347+0000 7fbd89e00640 1 --2- 172.21.15.16:0/1909139221 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd84155a20 0x7fbd840adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:46:10.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.348+0000 7fbd89400640 1 --2- 172.21.15.16:0/1909139221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd84155df0 0x7fbd840ad9b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009910 tx=0x7fbd740091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95bcf7247b198330 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:10.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.349+0000 7fbd88a00640 1 -- 172.21.15.16:0/1909139221 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7402ae40 con 0x7fbd84155df0 2024-08-17T02:46:10.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.349+0000 7fbd88a00640 1 -- 172.21.15.16:0/1909139221 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd74032440 con 0x7fbd84155df0 2024-08-17T02:46:10.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.349+0000 7fbd88a00640 1 -- 172.21.15.16:0/1909139221 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd740327e0 con 0x7fbd84155df0 2024-08-17T02:46:10.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.349+0000 7fbd8c800640 1 -- 172.21.15.16:0/1909139221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd84155df0 msgr2=0x7fbd840ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:10.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.349+0000 7fbd8c800640 1 --2- 172.21.15.16:0/1909139221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd84155df0 0x7fbd840ad9b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbd74009910 tx=0x7fbd740091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.349+0000 7fbd8c800640 1 -- 172.21.15.16:0/1909139221 shutdown_connections 2024-08-17T02:46:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.349+0000 7fbd8c800640 1 --2- 172.21.15.16:0/1909139221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd84155df0 0x7fbd840ad9b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.349+0000 7fbd8c800640 1 --2- 172.21.15.16:0/1909139221 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd84155a20 0x7fbd840adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.349+0000 7fbd8c800640 1 -- 172.21.15.16:0/1909139221 >> 172.21.15.16:0/1909139221 conn(0x7fbd840cb9b0 msgr2=0x7fbd840cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.349+0000 7fbd8c800640 1 -- 172.21.15.16:0/1909139221 shutdown_connections 2024-08-17T02:46:10.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.349+0000 7fbd8c800640 1 -- 172.21.15.16:0/1909139221 wait complete. 2024-08-17T02:46:10.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.351+0000 7fbd8c800640 1 Processor -- start 2024-08-17T02:46:10.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.351+0000 7fbd8c800640 1 -- start start 2024-08-17T02:46:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8c800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd8406d7f0 0x7fbd8414c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8c800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd84155a20 0x7fbd84151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8c800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd84155df0 0x7fbd841518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8c800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd841509b0 con 0x7fbd84155df0 2024-08-17T02:46:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8c800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8414c5c0 con 0x7fbd84155a20 2024-08-17T02:46:10.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8c800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8414fd30 con 0x7fbd8406d7f0 2024-08-17T02:46:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd84155df0 0x7fbd841518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd84155df0 0x7fbd841518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48444/0 (socket says 172.21.15.16:48444) 2024-08-17T02:46:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8a800640 1 -- 172.21.15.16:0/3576092459 learned_addr learned my addr 172.21.15.16:0/3576092459 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd89400640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd84155a20 0x7fbd84151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd89e00640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd8406d7f0 0x7fbd8414c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8a800640 1 -- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd8406d7f0 msgr2=0x7fbd8414c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:10.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8a800640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd8406d7f0 0x7fbd8414c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8a800640 1 -- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd84155a20 msgr2=0x7fbd84151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:10.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8a800640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd84155a20 0x7fbd84151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd8a800640 1 -- 172.21.15.16:0/3576092459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd78009390 con 0x7fbd84155df0 2024-08-17T02:46:10.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.352+0000 7fbd89400640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd84155a20 0x7fbd84151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:46:10.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.353+0000 7fbd8a800640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd84155df0 0x7fbd841518b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000c960 tx=0x7fbd800016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:10.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.353+0000 7fbd7d600640 1 -- 172.21.15.16:0/3576092459 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80005d60 con 0x7fbd84155df0 2024-08-17T02:46:10.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.353+0000 7fbd7d600640 1 -- 172.21.15.16:0/3576092459 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8000a640 con 0x7fbd84155df0 2024-08-17T02:46:10.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.353+0000 7fbd7d600640 1 -- 172.21.15.16:0/3576092459 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000aa00 con 0x7fbd84155df0 2024-08-17T02:46:10.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.353+0000 7fbd8c800640 1 -- 172.21.15.16:0/3576092459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd74009450 con 0x7fbd84155df0 2024-08-17T02:46:10.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.353+0000 7fbd8c800640 1 -- 172.21.15.16:0/3576092459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd841502f0 con 0x7fbd84155df0 2024-08-17T02:46:10.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.354+0000 7fbd8c800640 1 -- 172.21.15.16:0/3576092459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd84064db0 con 0x7fbd84155df0 2024-08-17T02:46:10.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.355+0000 7fbd7d600640 1 -- 172.21.15.16:0/3576092459 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fbd800070f0 con 0x7fbd84155df0 2024-08-17T02:46:10.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.356+0000 7fbd7d600640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbd68065150 0x7fbd68067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:10.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.356+0000 7fbd7d600640 1 -- 172.21.15.16:0/3576092459 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbd80081da0 con 0x7fbd84155df0 2024-08-17T02:46:10.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.356+0000 7fbd89e00640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbd68065150 0x7fbd68067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:10.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.357+0000 7fbd89e00640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbd68065150 0x7fbd68067600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009eb0 tx=0x7fbd780090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:10.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.360+0000 7fbd7d600640 1 -- 172.21.15.16:0/3576092459 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fbd80050240 con 0x7fbd84155df0 2024-08-17T02:46:10.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.610+0000 7fbd8c800640 1 -- 172.21.15.16:0/3576092459 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fbd84154690 con 0x7fbd84155df0 2024-08-17T02:46:10.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.614+0000 7fbd7d600640 1 -- 172.21.15.16:0/3576092459 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbd80053860 con 0x7fbd84155df0 2024-08-17T02:46:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.616+0000 7fbd8c800640 1 -- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbd68065150 msgr2=0x7fbd68067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.616+0000 7fbd8c800640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbd68065150 0x7fbd68067600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009eb0 tx=0x7fbd780090f0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.616+0000 7fbd8c800640 1 -- 172.21.15.16:0/3576092459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd84155df0 msgr2=0x7fbd841518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.616+0000 7fbd8c800640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd84155df0 0x7fbd841518b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000c960 tx=0x7fbd800016c0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.617+0000 7fbd8c800640 1 -- 172.21.15.16:0/3576092459 shutdown_connections 2024-08-17T02:46:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.617+0000 7fbd8c800640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbd68065150 0x7fbd68067600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.617+0000 7fbd8c800640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd84155df0 0x7fbd841518b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.617+0000 7fbd8c800640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd84155a20 0x7fbd84151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.617+0000 7fbd8c800640 1 --2- 172.21.15.16:0/3576092459 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbd8406d7f0 0x7fbd8414c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.617+0000 7fbd8c800640 1 -- 172.21.15.16:0/3576092459 >> 172.21.15.16:0/3576092459 conn(0x7fbd840cb9b0 msgr2=0x7fbd84153660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:10.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.617+0000 7fbd8c800640 1 -- 172.21.15.16:0/3576092459 shutdown_connections 2024-08-17T02:46:10.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.617+0000 7fbd8c800640 1 -- 172.21.15.16:0/3576092459 wait complete. 2024-08-17T02:46:10.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-08-17T02:46:10.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.786+0000 7fb761c00640 1 Processor -- start 2024-08-17T02:46:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.786+0000 7fb761c00640 1 -- start start 2024-08-17T02:46:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.787+0000 7fb761c00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb75c155550 0x7fb75c0aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.787+0000 7fb761c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb75c06e080 0x7fb75c06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.787+0000 7fb761c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb75c0afc10 con 0x7fb75c155180 2024-08-17T02:46:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.787+0000 7fb761c00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75c06ea00 con 0x7fb75c155550 2024-08-17T02:46:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.787+0000 7fb761c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c0af000 con 0x7fb75c06e080 2024-08-17T02:46:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.787+0000 7fb753e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb75c155550 0x7fb75c0aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.787+0000 7fb75b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb75c06e080 0x7fb75c06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.787+0000 7fb753e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb75c155550 0x7fb75c0aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48458/0 (socket says 172.21.15.16:48458) 2024-08-17T02:46:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.787+0000 7fb753e00640 1 -- 172.21.15.16:0/1894681449 learned_addr learned my addr 172.21.15.16:0/1894681449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.787+0000 7fb75aa00640 1 --1- 172.21.15.16:0/1894681449 >> v1:172.21.15.142:6789/0 conn(0x7fb75c155180 0x7fb75c0ae070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:44272/0 (socket says 172.21.15.16:44272) 2024-08-17T02:46:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.788+0000 7fb753e00640 1 -- 172.21.15.16:0/1894681449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb75c06e080 msgr2=0x7fb75c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.788+0000 7fb753e00640 1 --2- 172.21.15.16:0/1894681449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb75c06e080 0x7fb75c06e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.788+0000 7fb753e00640 1 -- 172.21.15.16:0/1894681449 >> v1:172.21.15.142:6789/0 conn(0x7fb75c155180 legacy=0x7fb75c0ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.788+0000 7fb75a000640 1 -- 172.21.15.16:0/1894681449 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377030929 0 0) 0x7fb75c0afc10 con 0x7fb75c155180 2024-08-17T02:46:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.788+0000 7fb753e00640 1 -- 172.21.15.16:0/1894681449 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c0af2e0 con 0x7fb75c155550 2024-08-17T02:46:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.789+0000 7fb753e00640 1 --2- 172.21.15.16:0/1894681449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb75c155550 0x7fb75c0aea20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb744009940 tx=0x7fb7440091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c92d0260322fb server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.789+0000 7fb75a000640 1 -- 172.21.15.16:0/1894681449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74402ae40 con 0x7fb75c155550 2024-08-17T02:46:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.789+0000 7fb75a000640 1 -- 172.21.15.16:0/1894681449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb744032440 con 0x7fb75c155550 2024-08-17T02:46:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.789+0000 7fb75a000640 1 -- 172.21.15.16:0/1894681449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7440327e0 con 0x7fb75c155550 2024-08-17T02:46:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.790+0000 7fb761c00640 1 -- 172.21.15.16:0/1894681449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb75c155550 msgr2=0x7fb75c0aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.790+0000 7fb761c00640 1 --2- 172.21.15.16:0/1894681449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb75c155550 0x7fb75c0aea20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb744009940 tx=0x7fb7440091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.790+0000 7fb761c00640 1 -- 172.21.15.16:0/1894681449 shutdown_connections 2024-08-17T02:46:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.790+0000 7fb761c00640 1 --2- 172.21.15.16:0/1894681449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb75c06e080 0x7fb75c06e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.790+0000 7fb761c00640 1 --2- 172.21.15.16:0/1894681449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb75c155550 0x7fb75c0aea20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.790+0000 7fb761c00640 1 -- 172.21.15.16:0/1894681449 >> 172.21.15.16:0/1894681449 conn(0x7fb75c0cb9b0 msgr2=0x7fb75c0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.790+0000 7fb761c00640 1 -- 172.21.15.16:0/1894681449 shutdown_connections 2024-08-17T02:46:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.790+0000 7fb761c00640 1 -- 172.21.15.16:0/1894681449 wait complete. 2024-08-17T02:46:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.791+0000 7fb761c00640 1 Processor -- start 2024-08-17T02:46:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.791+0000 7fb761c00640 1 -- start start 2024-08-17T02:46:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.792+0000 7fb761c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c06e080 0x7fb75c1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.792+0000 7fb761c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb75c155180 0x7fb75c169800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.792+0000 7fb761c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb75c155550 0x7fb75c16a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.792+0000 7fb761c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c156950 con 0x7fb75c155550 2024-08-17T02:46:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.792+0000 7fb761c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c169370 con 0x7fb75c06e080 2024-08-17T02:46:10.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.792+0000 7fb761c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c156390 con 0x7fb75c155180 2024-08-17T02:46:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.792+0000 7fb75b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb75c155550 0x7fb75c16a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.792+0000 7fb753e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb75c155180 0x7fb75c169800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.792+0000 7fb75aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c06e080 0x7fb75c1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.792+0000 7fb75b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb75c155550 0x7fb75c16a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48462/0 (socket says 172.21.15.16:48462) 2024-08-17T02:46:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.792+0000 7fb753e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb75c155180 0x7fb75c169800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:59522/0 (socket says 172.21.15.16:59522) 2024-08-17T02:46:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.792+0000 7fb75b400640 1 -- 172.21.15.16:0/790933759 learned_addr learned my addr 172.21.15.16:0/790933759 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.793+0000 7fb753e00640 1 -- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c06e080 msgr2=0x7fb75c1559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.793+0000 7fb753e00640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c06e080 0x7fb75c1559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.793+0000 7fb753e00640 1 -- 172.21.15.16:0/790933759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb75c155550 msgr2=0x7fb75c16a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.793+0000 7fb753e00640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb75c155550 0x7fb75c16a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:10.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.793+0000 7fb753e00640 1 -- 172.21.15.16:0/790933759 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb744009450 con 0x7fb75c155180 2024-08-17T02:46:10.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.793+0000 7fb75b400640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb75c155550 0x7fb75c16a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:46:10.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.793+0000 7fb753e00640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb75c155180 0x7fb75c169800 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb744031000 tx=0x7fb744032000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:10.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.793+0000 7fb752000640 1 -- 172.21.15.16:0/790933759 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744000d10 con 0x7fb75c155180 2024-08-17T02:46:10.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.793+0000 7fb752000640 1 -- 172.21.15.16:0/790933759 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb744031de0 con 0x7fb75c155180 2024-08-17T02:46:10.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.793+0000 7fb752000640 1 -- 172.21.15.16:0/790933759 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74403d670 con 0x7fb75c155180 2024-08-17T02:46:10.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.793+0000 7fb761c00640 1 -- 172.21.15.16:0/790933759 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75c156670 con 0x7fb75c155180 2024-08-17T02:46:10.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.794+0000 7fb761c00640 1 -- 172.21.15.16:0/790933759 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75c0c6600 con 0x7fb75c155180 2024-08-17T02:46:10.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.795+0000 7fb761c00640 1 -- 172.21.15.16:0/790933759 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75c168e60 con 0x7fb75c155180 2024-08-17T02:46:10.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.795+0000 7fb752000640 1 -- 172.21.15.16:0/790933759 <== mon.2 v2:172.21.15.142:3301/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fb744039040 con 0x7fb75c155180 2024-08-17T02:46:10.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.796+0000 7fb752000640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb738065150 0x7fb738067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:10.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.796+0000 7fb752000640 1 -- 172.21.15.16:0/790933759 <== mon.2 v2:172.21.15.142:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb74402ae40 con 0x7fb75c155180 2024-08-17T02:46:10.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.797+0000 7fb75aa00640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb738065150 0x7fb738067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:10.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.797+0000 7fb75aa00640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb738065150 0x7fb738067600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0ccaa0 tx=0x7fb74c007cc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:10.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:10.801+0000 7fb752000640 1 -- 172.21.15.16:0/790933759 <== mon.2 v2:172.21.15.142:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb75c168e60 con 0x7fb75c155180 2024-08-17T02:46:11.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.047+0000 7fb761c00640 1 -- 172.21.15.16:0/790933759 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fb75c168e60 con 0x7fb75c155180 2024-08-17T02:46:11.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.053+0000 7fb752000640 1 -- 172.21.15.16:0/790933759 <== mon.2 v2:172.21.15.142:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb75c168e60 con 0x7fb75c155180 2024-08-17T02:46:11.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.055+0000 7fb761c00640 1 -- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb738065150 msgr2=0x7fb738067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.055+0000 7fb761c00640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb738065150 0x7fb738067600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0ccaa0 tx=0x7fb74c007cc0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.055+0000 7fb761c00640 1 -- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb75c155180 msgr2=0x7fb75c169800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.055+0000 7fb761c00640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb75c155180 0x7fb75c169800 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb744031000 tx=0x7fb744032000 comp rx=0 tx=0).stop 2024-08-17T02:46:11.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.055+0000 7fb761c00640 1 -- 172.21.15.16:0/790933759 shutdown_connections 2024-08-17T02:46:11.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.055+0000 7fb761c00640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb738065150 0x7fb738067600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.055+0000 7fb761c00640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb75c155550 0x7fb75c16a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.055+0000 7fb761c00640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb75c155180 0x7fb75c169800 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.055+0000 7fb761c00640 1 --2- 172.21.15.16:0/790933759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb75c06e080 0x7fb75c1559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.055+0000 7fb761c00640 1 -- 172.21.15.16:0/790933759 >> 172.21.15.16:0/790933759 conn(0x7fb75c0cb9b0 msgr2=0x7fb75c06a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:11.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.055+0000 7fb761c00640 1 -- 172.21.15.16:0/790933759 shutdown_connections 2024-08-17T02:46:11.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.055+0000 7fb761c00640 1 -- 172.21.15.16:0/790933759 wait complete. 2024-08-17T02:46:11.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-08-17T02:46:11.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.222+0000 7f66c4200640 1 Processor -- start 2024-08-17T02:46:11.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.222+0000 7f66c4200640 1 -- start start 2024-08-17T02:46:11.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.223+0000 7f66c4200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66bc155180 0x7f66bc06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:11.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.223+0000 7f66c4200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66bc155600 0x7f66bc06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:11.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.223+0000 7f66c4200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66bc0afc10 con 0x7f66bc0aea20 2024-08-17T02:46:11.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.223+0000 7f66c4200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66bc0aedf0 con 0x7f66bc155180 2024-08-17T02:46:11.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.223+0000 7f66c4200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0af000 con 0x7f66bc155600 2024-08-17T02:46:11.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.224+0000 7f66c0e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66bc155180 0x7f66bc06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:11.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.224+0000 7f66c1800640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7f66bc0aea20 0x7f66bc0ae070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:44282/0 (socket says 172.21.15.16:44282) 2024-08-17T02:46:11.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.224+0000 7f66c1800640 1 -- 172.21.15.16:0/4178934963 learned_addr learned my addr 172.21.15.16:0/4178934963 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:11.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.224+0000 7f66c0e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66bc155180 0x7f66bc06d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48472/0 (socket says 172.21.15.16:48472) 2024-08-17T02:46:11.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.224+0000 7f66c2200640 1 --2- 172.21.15.16:0/4178934963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66bc155600 0x7f66bc06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:11.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.224+0000 7f66bbe00640 1 -- 172.21.15.16:0/4178934963 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267413523 0 0) 0x7f66bc0afc10 con 0x7f66bc0aea20 2024-08-17T02:46:11.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.224+0000 7f66c0e00640 1 -- 172.21.15.16:0/4178934963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66bc155600 msgr2=0x7f66bc06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:11.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.224+0000 7f66c0e00640 1 --2- 172.21.15.16:0/4178934963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66bc155600 0x7f66bc06dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.224+0000 7f66c0e00640 1 -- 172.21.15.16:0/4178934963 >> v1:172.21.15.142:6789/0 conn(0x7f66bc0aea20 legacy=0x7f66bc0ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.224+0000 7f66c0e00640 1 -- 172.21.15.16:0/4178934963 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc0af2e0 con 0x7f66bc155180 2024-08-17T02:46:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.225+0000 7f66c0e00640 1 --2- 172.21.15.16:0/4178934963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc155180 0x7f66bc06d7f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009850 tx=0x7f66a80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3102b8b98f5107b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.225+0000 7f66bbe00640 1 -- 172.21.15.16:0/4178934963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a802ad90 con 0x7f66bc155180 2024-08-17T02:46:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.226+0000 7f66bbe00640 1 -- 172.21.15.16:0/4178934963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a8032440 con 0x7f66bc155180 2024-08-17T02:46:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.226+0000 7f66bbe00640 1 -- 172.21.15.16:0/4178934963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a80327e0 con 0x7f66bc155180 2024-08-17T02:46:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.226+0000 7f66c4200640 1 -- 172.21.15.16:0/4178934963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc155180 msgr2=0x7f66bc06d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.226+0000 7f66c4200640 1 --2- 172.21.15.16:0/4178934963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc155180 0x7f66bc06d7f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009850 tx=0x7f66a80090f0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.226+0000 7f66c4200640 1 -- 172.21.15.16:0/4178934963 shutdown_connections 2024-08-17T02:46:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.226+0000 7f66c4200640 1 --2- 172.21.15.16:0/4178934963 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66bc155600 0x7f66bc06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.226+0000 7f66c4200640 1 --2- 172.21.15.16:0/4178934963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc155180 0x7f66bc06d7f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.226+0000 7f66c4200640 1 -- 172.21.15.16:0/4178934963 >> 172.21.15.16:0/4178934963 conn(0x7f66bc0cb9b0 msgr2=0x7f66bc0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.226+0000 7f66c4200640 1 -- 172.21.15.16:0/4178934963 shutdown_connections 2024-08-17T02:46:11.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.226+0000 7f66c4200640 1 -- 172.21.15.16:0/4178934963 wait complete. 2024-08-17T02:46:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.228+0000 7f66c4200640 1 Processor -- start 2024-08-17T02:46:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.228+0000 7f66c4200640 1 -- start start 2024-08-17T02:46:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.228+0000 7f66c4200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0aea20 0x7f66bc1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c4200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66bc155180 0x7f66bc0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c4200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66bc155600 0x7f66bc0b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c4200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0a54d0 con 0x7f66bc0aea20 2024-08-17T02:46:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c4200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0a6330 con 0x7f66bc155180 2024-08-17T02:46:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c4200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc0a5720 con 0x7f66bc155600 2024-08-17T02:46:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0aea20 0x7f66bc1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:11.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c1800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0aea20 0x7f66bc1563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48480/0 (socket says 172.21.15.16:48480) 2024-08-17T02:46:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c1800640 1 -- 172.21.15.16:0/3899023774 learned_addr learned my addr 172.21.15.16:0/3899023774 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c0e00640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66bc155180 0x7f66bc0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c2200640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66bc155600 0x7f66bc0b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c1800640 1 -- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66bc155600 msgr2=0x7f66bc0b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c1800640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66bc155600 0x7f66bc0b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c1800640 1 -- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66bc155180 msgr2=0x7f66bc0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c1800640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66bc155180 0x7f66bc0b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c1800640 1 -- 172.21.15.16:0/3899023774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a8009390 con 0x7f66bc0aea20 2024-08-17T02:46:11.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c0e00640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66bc155180 0x7f66bc0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:11.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.229+0000 7f66c1800640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0aea20 0x7f66bc1563d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c5c0 tx=0x7f66b0007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:11.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.230+0000 7f66b8c00640 1 -- 172.21.15.16:0/3899023774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b0005aa0 con 0x7f66bc0aea20 2024-08-17T02:46:11.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.230+0000 7f66b8c00640 1 -- 172.21.15.16:0/3899023774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b0005c00 con 0x7f66bc0aea20 2024-08-17T02:46:11.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.230+0000 7f66b8c00640 1 -- 172.21.15.16:0/3899023774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b00070f0 con 0x7f66bc0aea20 2024-08-17T02:46:11.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.230+0000 7f66c4200640 1 -- 172.21.15.16:0/3899023774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66bc0a5a00 con 0x7f66bc0aea20 2024-08-17T02:46:11.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.230+0000 7f66c4200640 1 -- 172.21.15.16:0/3899023774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc0a5fc0 con 0x7f66bc0aea20 2024-08-17T02:46:11.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.231+0000 7f66c4200640 1 -- 172.21.15.16:0/3899023774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66bc064db0 con 0x7f66bc0aea20 2024-08-17T02:46:11.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.232+0000 7f66b8c00640 1 -- 172.21.15.16:0/3899023774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f66b0007520 con 0x7f66bc0aea20 2024-08-17T02:46:11.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.233+0000 7f66b8c00640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f669c065150 0x7f669c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:11.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.233+0000 7f66b8c00640 1 -- 172.21.15.16:0/3899023774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f66b0081c00 con 0x7f66bc0aea20 2024-08-17T02:46:11.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.233+0000 7f66c0e00640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f669c065150 0x7f669c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:11.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.234+0000 7f66c0e00640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f669c065150 0x7f669c067600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f66a8034000 tx=0x7f66a8032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:11.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.237+0000 7f66b8c00640 1 -- 172.21.15.16:0/3899023774 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f66b004fff0 con 0x7f66bc0aea20 2024-08-17T02:46:11.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.484+0000 7f66c4200640 1 -- 172.21.15.16:0/3899023774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f66bc158bb0 con 0x7f66bc0aea20 2024-08-17T02:46:11.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.488+0000 7f66b8c00640 1 -- 172.21.15.16:0/3899023774 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f66b0053610 con 0x7f66bc0aea20 2024-08-17T02:46:11.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.490+0000 7f66c4200640 1 -- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f669c065150 msgr2=0x7f669c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.490+0000 7f66c4200640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f669c065150 0x7f669c067600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f66a8034000 tx=0x7f66a8032000 comp rx=0 tx=0).stop 2024-08-17T02:46:11.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.490+0000 7f66c4200640 1 -- 172.21.15.16:0/3899023774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0aea20 msgr2=0x7f66bc1563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.490+0000 7f66c4200640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0aea20 0x7f66bc1563d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f66b000c5c0 tx=0x7f66b0007d40 comp rx=0 tx=0).stop 2024-08-17T02:46:11.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.490+0000 7f66c4200640 1 -- 172.21.15.16:0/3899023774 shutdown_connections 2024-08-17T02:46:11.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.490+0000 7f66c4200640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f669c065150 0x7f669c067600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.490+0000 7f66c4200640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f66bc155600 0x7f66bc0b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.490+0000 7f66c4200640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f66bc155180 0x7f66bc0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.490+0000 7f66c4200640 1 --2- 172.21.15.16:0/3899023774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66bc0aea20 0x7f66bc1563d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.490+0000 7f66c4200640 1 -- 172.21.15.16:0/3899023774 >> 172.21.15.16:0/3899023774 conn(0x7f66bc0cb9b0 msgr2=0x7f66bc06e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:11.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.491+0000 7f66c4200640 1 -- 172.21.15.16:0/3899023774 shutdown_connections 2024-08-17T02:46:11.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.491+0000 7f66c4200640 1 -- 172.21.15.16:0/3899023774 wait complete. 2024-08-17T02:46:11.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-08-17T02:46:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.658+0000 7f8368200640 1 Processor -- start 2024-08-17T02:46:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.658+0000 7f8368200640 1 -- start start 2024-08-17T02:46:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.659+0000 7f8368200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f83600b1440 0x7f83600caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.659+0000 7f8368200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83600caa80 con 0x7f8360140580 2024-08-17T02:46:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.659+0000 7f8368200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83600c9e70 con 0x7f83601401b0 2024-08-17T02:46:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.659+0000 7f8368200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83600cb450 con 0x7f83600b1440 2024-08-17T02:46:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.659+0000 7f8366200640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8360140580 0x7f83600c9040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57006/0 (socket says 172.21.15.16:57006) 2024-08-17T02:46:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.659+0000 7f8366200640 1 -- 172.21.15.16:0/3366491988 learned_addr learned my addr 172.21.15.16:0/3366491988 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.660+0000 7f8364e00640 1 --2- 172.21.15.16:0/3366491988 >> v2:172.21.15.142:3300/0 conn(0x7f83600b1440 0x7f83600caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.660+0000 7f835fe00640 1 -- 172.21.15.16:0/3366491988 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791574074 0 0) 0x7f83600caa80 con 0x7f8360140580 2024-08-17T02:46:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.660+0000 7f835fe00640 1 -- 172.21.15.16:0/3366491988 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83480034e0 con 0x7f8360140580 2024-08-17T02:46:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.660+0000 7f8364e00640 1 -- 172.21.15.16:0/3366491988 >> v1:172.21.15.142:6789/0 conn(0x7f83601401b0 legacy=0x7f83600b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.660+0000 7f8364e00640 1 -- 172.21.15.16:0/3366491988 >> v1:172.21.15.16:6789/0 conn(0x7f8360140580 legacy=0x7f83600c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.660+0000 7f8364e00640 1 -- 172.21.15.16:0/3366491988 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83600ca150 con 0x7f83600b1440 2024-08-17T02:46:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.661+0000 7f8364e00640 1 --2- 172.21.15.16:0/3366491988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83600b1440 0x7f83600caf10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f834c009850 tx=0x7f834c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cde054e2ce4c1649 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.661+0000 7f835fe00640 1 -- 172.21.15.16:0/3366491988 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c02ae20 con 0x7f83600b1440 2024-08-17T02:46:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.661+0000 7f835fe00640 1 -- 172.21.15.16:0/3366491988 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c032440 con 0x7f83600b1440 2024-08-17T02:46:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.661+0000 7f835fe00640 1 -- 172.21.15.16:0/3366491988 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c0327e0 con 0x7f83600b1440 2024-08-17T02:46:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.662+0000 7f8368200640 1 -- 172.21.15.16:0/3366491988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83600b1440 msgr2=0x7f83600caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.662+0000 7f8368200640 1 --2- 172.21.15.16:0/3366491988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83600b1440 0x7f83600caf10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f834c009850 tx=0x7f834c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.662+0000 7f8368200640 1 -- 172.21.15.16:0/3366491988 shutdown_connections 2024-08-17T02:46:11.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.662+0000 7f8368200640 1 --2- 172.21.15.16:0/3366491988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83600b1440 0x7f83600caf10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.662+0000 7f8368200640 1 -- 172.21.15.16:0/3366491988 >> 172.21.15.16:0/3366491988 conn(0x7f83600cb9b0 msgr2=0x7f83600cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:11.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.662+0000 7f8368200640 1 -- 172.21.15.16:0/3366491988 shutdown_connections 2024-08-17T02:46:11.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.662+0000 7f8368200640 1 -- 172.21.15.16:0/3366491988 wait complete. 2024-08-17T02:46:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.663+0000 7f8368200640 1 Processor -- start 2024-08-17T02:46:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.664+0000 7f8368200640 1 -- start start 2024-08-17T02:46:11.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.664+0000 7f8368200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83600b1440 0x7f836016c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.664+0000 7f8368200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83601401b0 0x7f83600a8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.664+0000 7f8368200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8360140580 0x7f83600c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.664+0000 7f8368200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836016d270 con 0x7f8360140580 2024-08-17T02:46:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.664+0000 7f8368200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83600c73a0 con 0x7f83600b1440 2024-08-17T02:46:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.664+0000 7f8368200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836016ccb0 con 0x7f83601401b0 2024-08-17T02:46:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.665+0000 7f8366200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8360140580 0x7f83600c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.665+0000 7f8365800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83600b1440 0x7f836016c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.665+0000 7f8366200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8360140580 0x7f83600c7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48490/0 (socket says 172.21.15.16:48490) 2024-08-17T02:46:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.665+0000 7f8366200640 1 -- 172.21.15.16:0/2035984986 learned_addr learned my addr 172.21.15.16:0/2035984986 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.665+0000 7f8364e00640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83601401b0 0x7f83600a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.665+0000 7f8366200640 1 -- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83601401b0 msgr2=0x7f83600a8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.665+0000 7f8366200640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83601401b0 0x7f83600a8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.665+0000 7f8366200640 1 -- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83600b1440 msgr2=0x7f836016c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.665+0000 7f8366200640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83600b1440 0x7f836016c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.665+0000 7f8366200640 1 -- 172.21.15.16:0/2035984986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c009390 con 0x7f8360140580 2024-08-17T02:46:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.665+0000 7f8364e00640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83601401b0 0x7f83600a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:46:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.665+0000 7f8366200640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8360140580 0x7f83600c7830 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f835800cca0 tx=0x7f8358001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.666+0000 7f835cc00640 1 -- 172.21.15.16:0/2035984986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835800a850 con 0x7f8360140580 2024-08-17T02:46:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.666+0000 7f835cc00640 1 -- 172.21.15.16:0/2035984986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835800a9b0 con 0x7f8360140580 2024-08-17T02:46:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.666+0000 7f835cc00640 1 -- 172.21.15.16:0/2035984986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835800ad70 con 0x7f8360140580 2024-08-17T02:46:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.666+0000 7f8368200640 1 -- 172.21.15.16:0/2035984986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836016cf90 con 0x7f8360140580 2024-08-17T02:46:11.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.666+0000 7f8368200640 1 -- 172.21.15.16:0/2035984986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836016c910 con 0x7f8360140580 2024-08-17T02:46:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.667+0000 7f8368200640 1 -- 172.21.15.16:0/2035984986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8360064db0 con 0x7f8360140580 2024-08-17T02:46:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.668+0000 7f835cc00640 1 -- 172.21.15.16:0/2035984986 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f8358007300 con 0x7f8360140580 2024-08-17T02:46:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.669+0000 7f835cc00640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8340065150 0x7f8340067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.669+0000 7f835cc00640 1 -- 172.21.15.16:0/2035984986 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8358082130 con 0x7f8360140580 2024-08-17T02:46:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.669+0000 7f8365800640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8340065150 0x7f8340067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:11.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.670+0000 7f8365800640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8340065150 0x7f8340067600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8354007fd0 tx=0x7f8354007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:11.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.673+0000 7f835cc00640 1 -- 172.21.15.16:0/2035984986 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8358050670 con 0x7f8360140580 2024-08-17T02:46:11.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.922+0000 7f8368200640 1 -- 172.21.15.16:0/2035984986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f83600c74e0 con 0x7f8360140580 2024-08-17T02:46:11.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.926+0000 7f835cc00640 1 -- 172.21.15.16:0/2035984986 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8358053c90 con 0x7f8360140580 2024-08-17T02:46:11.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.928+0000 7f8368200640 1 -- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8340065150 msgr2=0x7f8340067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.928+0000 7f8368200640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8340065150 0x7f8340067600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8354007fd0 tx=0x7f8354007e30 comp rx=0 tx=0).stop 2024-08-17T02:46:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.928+0000 7f8368200640 1 -- 172.21.15.16:0/2035984986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8360140580 msgr2=0x7f83600c7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.928+0000 7f8368200640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8360140580 0x7f83600c7830 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f835800cca0 tx=0x7f8358001ae0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.928+0000 7f8368200640 1 -- 172.21.15.16:0/2035984986 shutdown_connections 2024-08-17T02:46:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.928+0000 7f8368200640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8340065150 0x7f8340067600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.928+0000 7f8368200640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8360140580 0x7f83600c7830 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.928+0000 7f8368200640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f83601401b0 0x7f83600a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.928+0000 7f8368200640 1 --2- 172.21.15.16:0/2035984986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83600b1440 0x7f836016c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.928+0000 7f8368200640 1 -- 172.21.15.16:0/2035984986 >> 172.21.15.16:0/2035984986 conn(0x7f83600cb9b0 msgr2=0x7f8360069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.928+0000 7f8368200640 1 -- 172.21.15.16:0/2035984986 shutdown_connections 2024-08-17T02:46:11.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:11.928+0000 7f8368200640 1 -- 172.21.15.16:0/2035984986 wait complete. 2024-08-17T02:46:11.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-08-17T02:46:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.095+0000 7f8befe00640 1 Processor -- start 2024-08-17T02:46:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.095+0000 7f8befe00640 1 -- start start 2024-08-17T02:46:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.096+0000 7f8befe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bf0156ad0 0x7f8bf00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.096+0000 7f8befe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bf0156ea0 0x7f8bf006d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.096+0000 7f8befe00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf00caaf0 con 0x7f8bf006dab0 2024-08-17T02:46:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.096+0000 7f8befe00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf00ca900 con 0x7f8bf0156ad0 2024-08-17T02:46:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.096+0000 7f8befe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00cc560 con 0x7f8bf0156ea0 2024-08-17T02:46:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.096+0000 7f8beea00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bf0156ad0 0x7f8bf00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.096+0000 7f8beea00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bf0156ad0 0x7f8bf00cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48500/0 (socket says 172.21.15.16:48500) 2024-08-17T02:46:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.096+0000 7f8bef400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8bf006dab0 0x7f8bf00c8e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57008/0 (socket says 172.21.15.16:57008) 2024-08-17T02:46:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.096+0000 7f8beea00640 1 -- 172.21.15.16:0/3666486174 learned_addr learned my addr 172.21.15.16:0/3666486174 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.096+0000 7f8be7e00640 1 --2- 172.21.15.16:0/3666486174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bf0156ea0 0x7f8bf006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.097+0000 7f8beea00640 1 -- 172.21.15.16:0/3666486174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bf0156ea0 msgr2=0x7f8bf006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.097+0000 7f8beea00640 1 --2- 172.21.15.16:0/3666486174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bf0156ea0 0x7f8bf006d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.097+0000 7f8beea00640 1 -- 172.21.15.16:0/3666486174 >> v1:172.21.15.16:6789/0 conn(0x7f8bf006dab0 legacy=0x7f8bf00c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.097+0000 7f8beea00640 1 -- 172.21.15.16:0/3666486174 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf00cad90 con 0x7f8bf0156ad0 2024-08-17T02:46:12.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.097+0000 7f8bee000640 1 -- 172.21.15.16:0/3666486174 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934350451 0 0) 0x7f8bf00caaf0 con 0x7f8bf006dab0 2024-08-17T02:46:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.098+0000 7f8beea00640 1 --2- 172.21.15.16:0/3666486174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf0156ad0 0x7f8bf00cbea0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009940 tx=0x7f8be00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e359a3c096ca02a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.098+0000 7f8bee000640 1 -- 172.21.15.16:0/3666486174 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be002ae40 con 0x7f8bf0156ad0 2024-08-17T02:46:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.098+0000 7f8bee000640 1 -- 172.21.15.16:0/3666486174 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be0032440 con 0x7f8bf0156ad0 2024-08-17T02:46:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.098+0000 7f8bee000640 1 -- 172.21.15.16:0/3666486174 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be00327e0 con 0x7f8bf0156ad0 2024-08-17T02:46:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.099+0000 7f8befe00640 1 -- 172.21.15.16:0/3666486174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf0156ad0 msgr2=0x7f8bf00cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.099+0000 7f8befe00640 1 --2- 172.21.15.16:0/3666486174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf0156ad0 0x7f8bf00cbea0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009940 tx=0x7f8be00091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.099+0000 7f8befe00640 1 -- 172.21.15.16:0/3666486174 shutdown_connections 2024-08-17T02:46:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.099+0000 7f8befe00640 1 --2- 172.21.15.16:0/3666486174 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bf0156ea0 0x7f8bf006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.099+0000 7f8befe00640 1 --2- 172.21.15.16:0/3666486174 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf0156ad0 0x7f8bf00cbea0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.099+0000 7f8befe00640 1 -- 172.21.15.16:0/3666486174 >> 172.21.15.16:0/3666486174 conn(0x7f8bf0067750 msgr2=0x7f8bf00cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.099+0000 7f8befe00640 1 -- 172.21.15.16:0/3666486174 shutdown_connections 2024-08-17T02:46:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.099+0000 7f8befe00640 1 -- 172.21.15.16:0/3666486174 wait complete. 2024-08-17T02:46:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.100+0000 7f8befe00640 1 Processor -- start 2024-08-17T02:46:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.100+0000 7f8befe00640 1 -- start start 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.101+0000 7f8befe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf006dab0 0x7f8bf0144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.101+0000 7f8befe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf0156ad0 0x7f8bf014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.101+0000 7f8befe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bf0156ea0 0x7f8bf0152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.101+0000 7f8befe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf014d3e0 con 0x7f8bf006dab0 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.101+0000 7f8befe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf014d6c0 con 0x7f8bf0156ad0 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.101+0000 7f8befe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf014d9a0 con 0x7f8bf0156ea0 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.101+0000 7f8beea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf006dab0 0x7f8bf0144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.101+0000 7f8beea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf006dab0 0x7f8bf0144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48506/0 (socket says 172.21.15.16:48506) 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.101+0000 7f8beea00640 1 -- 172.21.15.16:0/1587461809 learned_addr learned my addr 172.21.15.16:0/1587461809 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.101+0000 7f8bef400640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bf0156ea0 0x7f8bf0152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.102+0000 7f8be7e00640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf0156ad0 0x7f8bf014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.102+0000 7f8beea00640 1 -- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bf0156ea0 msgr2=0x7f8bf0152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.102+0000 7f8beea00640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bf0156ea0 0x7f8bf0152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.102+0000 7f8beea00640 1 -- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf0156ad0 msgr2=0x7f8bf014e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.102+0000 7f8beea00640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf0156ad0 0x7f8bf014e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.102+0000 7f8beea00640 1 -- 172.21.15.16:0/1587461809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be0009450 con 0x7f8bf006dab0 2024-08-17T02:46:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.102+0000 7f8beea00640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf006dab0 0x7f8bf0144320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8be0035040 tx=0x7f8be0032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.102+0000 7f8be6000640 1 -- 172.21.15.16:0/1587461809 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0000d10 con 0x7f8bf006dab0 2024-08-17T02:46:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.102+0000 7f8be6000640 1 -- 172.21.15.16:0/1587461809 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be0031de0 con 0x7f8bf006dab0 2024-08-17T02:46:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.102+0000 7f8be6000640 1 -- 172.21.15.16:0/1587461809 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be003d690 con 0x7f8bf006dab0 2024-08-17T02:46:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.102+0000 7f8befe00640 1 -- 172.21.15.16:0/1587461809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf014dc80 con 0x7f8bf006dab0 2024-08-17T02:46:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.102+0000 7f8befe00640 1 -- 172.21.15.16:0/1587461809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf014cf50 con 0x7f8bf006dab0 2024-08-17T02:46:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.103+0000 7f8befe00640 1 -- 172.21.15.16:0/1587461809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf014d140 con 0x7f8bf006dab0 2024-08-17T02:46:12.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.104+0000 7f8be6000640 1 -- 172.21.15.16:0/1587461809 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f8be0039040 con 0x7f8bf006dab0 2024-08-17T02:46:12.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.105+0000 7f8be6000640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8bcc065180 0x7f8bcc067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.105+0000 7f8be6000640 1 -- 172.21.15.16:0/1587461809 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8be002ae40 con 0x7f8bf006dab0 2024-08-17T02:46:12.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.105+0000 7f8be7e00640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8bcc065180 0x7f8bcc067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.106+0000 7f8be7e00640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8bcc065180 0x7f8bcc067630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00ad700 tx=0x7f8bd8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:12.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.109+0000 7f8be6000640 1 -- 172.21.15.16:0/1587461809 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8be0070770 con 0x7f8bf006dab0 2024-08-17T02:46:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.357+0000 7f8befe00640 1 -- 172.21.15.16:0/1587461809 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f8bf0071050 con 0x7f8bf006dab0 2024-08-17T02:46:12.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.360+0000 7f8be6000640 1 -- 172.21.15.16:0/1587461809 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8be0073d90 con 0x7f8bf006dab0 2024-08-17T02:46:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.362+0000 7f8befe00640 1 -- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8bcc065180 msgr2=0x7f8bcc067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.362+0000 7f8befe00640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8bcc065180 0x7f8bcc067630 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00ad700 tx=0x7f8bd8008040 comp rx=0 tx=0).stop 2024-08-17T02:46:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.362+0000 7f8befe00640 1 -- 172.21.15.16:0/1587461809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf006dab0 msgr2=0x7f8bf0144320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.362+0000 7f8befe00640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf006dab0 0x7f8bf0144320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8be0035040 tx=0x7f8be0032000 comp rx=0 tx=0).stop 2024-08-17T02:46:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.362+0000 7f8befe00640 1 -- 172.21.15.16:0/1587461809 shutdown_connections 2024-08-17T02:46:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.362+0000 7f8befe00640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8bcc065180 0x7f8bcc067630 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.362+0000 7f8befe00640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8bf0156ea0 0x7f8bf0152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.362+0000 7f8befe00640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bf0156ad0 0x7f8bf014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.362+0000 7f8befe00640 1 --2- 172.21.15.16:0/1587461809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bf006dab0 0x7f8bf0144320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.362+0000 7f8befe00640 1 -- 172.21.15.16:0/1587461809 >> 172.21.15.16:0/1587461809 conn(0x7f8bf0067750 msgr2=0x7f8bf00c8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.363+0000 7f8befe00640 1 -- 172.21.15.16:0/1587461809 shutdown_connections 2024-08-17T02:46:12.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.363+0000 7f8befe00640 1 -- 172.21.15.16:0/1587461809 wait complete. 2024-08-17T02:46:12.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-08-17T02:46:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.531+0000 7f6e47600640 1 Processor -- start 2024-08-17T02:46:12.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.531+0000 7f6e47600640 1 -- start start 2024-08-17T02:46:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.532+0000 7f6e47600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e40155a20 0x7f6e400adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.532+0000 7f6e47600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f6e40155df0 0x7f6e400ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.532+0000 7f6e47600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e40156670 con 0x7f6e4006d7f0 2024-08-17T02:46:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.532+0000 7f6e47600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e400b0350 con 0x7f6e40155a20 2024-08-17T02:46:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.532+0000 7f6e47600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4006e5e0 con 0x7f6e40155df0 2024-08-17T02:46:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.532+0000 7f6e44c00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e40155a20 0x7f6e400adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.532+0000 7f6e44c00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e40155a20 0x7f6e400adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48522/0 (socket says 172.21.15.16:48522) 2024-08-17T02:46:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.532+0000 7f6e45600640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6e4006d7f0 0x7f6e400ac790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57012/0 (socket says 172.21.15.16:57012) 2024-08-17T02:46:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.532+0000 7f6e44c00640 1 -- 172.21.15.16:0/3830028083 learned_addr learned my addr 172.21.15.16:0/3830028083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:12.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.533+0000 7f6e37e00640 1 --2- 172.21.15.16:0/3830028083 >> v2:172.21.15.142:3300/0 conn(0x7f6e40155df0 0x7f6e400ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.533+0000 7f6e44c00640 1 -- 172.21.15.16:0/3830028083 >> v2:172.21.15.142:3300/0 conn(0x7f6e40155df0 msgr2=0x7f6e400ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.533+0000 7f6e44c00640 1 --2- 172.21.15.16:0/3830028083 >> v2:172.21.15.142:3300/0 conn(0x7f6e40155df0 0x7f6e400ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.533+0000 7f6e44c00640 1 -- 172.21.15.16:0/3830028083 >> v1:172.21.15.16:6789/0 conn(0x7f6e4006d7f0 legacy=0x7f6e400ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.533+0000 7f6e3fe00640 1 -- 172.21.15.16:0/3830028083 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776679695 0 0) 0x7f6e40156670 con 0x7f6e4006d7f0 2024-08-17T02:46:12.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.533+0000 7f6e44c00640 1 -- 172.21.15.16:0/3830028083 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4006e8c0 con 0x7f6e40155a20 2024-08-17T02:46:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.534+0000 7f6e44c00640 1 --2- 172.21.15.16:0/3830028083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e40155a20 0x7f6e400adf10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6e4006aa10 tx=0x7f6e300091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362d90567a08b760 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:12.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.534+0000 7f6e3fe00640 1 -- 172.21.15.16:0/3830028083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3002ae80 con 0x7f6e40155a20 2024-08-17T02:46:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.534+0000 7f6e3fe00640 1 -- 172.21.15.16:0/3830028083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e30032440 con 0x7f6e40155a20 2024-08-17T02:46:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.534+0000 7f6e3fe00640 1 -- 172.21.15.16:0/3830028083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e300327c0 con 0x7f6e40155a20 2024-08-17T02:46:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.534+0000 7f6e47600640 1 -- 172.21.15.16:0/3830028083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e40155a20 msgr2=0x7f6e400adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.534+0000 7f6e47600640 1 --2- 172.21.15.16:0/3830028083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e40155a20 0x7f6e400adf10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6e4006aa10 tx=0x7f6e300091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.535+0000 7f6e47600640 1 -- 172.21.15.16:0/3830028083 shutdown_connections 2024-08-17T02:46:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.535+0000 7f6e47600640 1 --2- 172.21.15.16:0/3830028083 >> v2:172.21.15.142:3300/0 conn(0x7f6e40155df0 0x7f6e400ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.535+0000 7f6e47600640 1 --2- 172.21.15.16:0/3830028083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e40155a20 0x7f6e400adf10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.535+0000 7f6e47600640 1 -- 172.21.15.16:0/3830028083 >> 172.21.15.16:0/3830028083 conn(0x7f6e400cb9b0 msgr2=0x7f6e400cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.535+0000 7f6e47600640 1 -- 172.21.15.16:0/3830028083 shutdown_connections 2024-08-17T02:46:12.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.535+0000 7f6e47600640 1 -- 172.21.15.16:0/3830028083 wait complete. 2024-08-17T02:46:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.536+0000 7f6e47600640 1 Processor -- start 2024-08-17T02:46:12.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.536+0000 7f6e47600640 1 -- start start 2024-08-17T02:46:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.537+0000 7f6e47600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4006d7f0 0x7f6e400adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.537+0000 7f6e47600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e40155a20 0x7f6e40169fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.537+0000 7f6e47600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e40155df0 0x7f6e4013db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.537+0000 7f6e47600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400ae410 con 0x7f6e4006d7f0 2024-08-17T02:46:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.537+0000 7f6e47600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400a6640 con 0x7f6e40155a20 2024-08-17T02:46:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.537+0000 7f6e47600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e400a6920 con 0x7f6e40155df0 2024-08-17T02:46:12.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.537+0000 7f6e44c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4006d7f0 0x7f6e400adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.537+0000 7f6e44c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4006d7f0 0x7f6e400adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48524/0 (socket says 172.21.15.16:48524) 2024-08-17T02:46:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.537+0000 7f6e44c00640 1 -- 172.21.15.16:0/1393328581 learned_addr learned my addr 172.21.15.16:0/1393328581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e37e00640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e40155a20 0x7f6e40169fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e44c00640 1 -- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e40155df0 msgr2=0x7f6e4013db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e44c00640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e40155df0 0x7f6e4013db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e44c00640 1 -- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e40155a20 msgr2=0x7f6e40169fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e44c00640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e40155a20 0x7f6e40169fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e44c00640 1 -- 172.21.15.16:0/1393328581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e30009450 con 0x7f6e4006d7f0 2024-08-17T02:46:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e37e00640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e40155a20 0x7f6e40169fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:46:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e44c00640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4006d7f0 0x7f6e400adb70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6e30035040 tx=0x7f6e30032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e3cc00640 1 -- 172.21.15.16:0/1393328581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e30000d10 con 0x7f6e4006d7f0 2024-08-17T02:46:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e3cc00640 1 -- 172.21.15.16:0/1393328581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e30031de0 con 0x7f6e4006d7f0 2024-08-17T02:46:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e3cc00640 1 -- 172.21.15.16:0/1393328581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3003d670 con 0x7f6e4006d7f0 2024-08-17T02:46:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e47600640 1 -- 172.21.15.16:0/1393328581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e400a6c00 con 0x7f6e4006d7f0 2024-08-17T02:46:12.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.538+0000 7f6e47600640 1 -- 172.21.15.16:0/1393328581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e400ae0b0 con 0x7f6e4006d7f0 2024-08-17T02:46:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.540+0000 7f6e47600640 1 -- 172.21.15.16:0/1393328581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e40064db0 con 0x7f6e4006d7f0 2024-08-17T02:46:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.540+0000 7f6e3cc00640 1 -- 172.21.15.16:0/1393328581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f6e30039040 con 0x7f6e4006d7f0 2024-08-17T02:46:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.542+0000 7f6e3cc00640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6e1c065150 0x7f6e1c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.542+0000 7f6e3cc00640 1 -- 172.21.15.16:0/1393328581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6e300a2160 con 0x7f6e4006d7f0 2024-08-17T02:46:12.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.542+0000 7f6e37e00640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6e1c065150 0x7f6e1c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.543+0000 7f6e37e00640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6e1c065150 0x7f6e1c067600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e40069430 tx=0x7f6e28008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.546+0000 7f6e3cc00640 1 -- 172.21.15.16:0/1393328581 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6e300704d0 con 0x7f6e4006d7f0 2024-08-17T02:46:12.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.795+0000 7f6e47600640 1 -- 172.21.15.16:0/1393328581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f6e40158d60 con 0x7f6e4006d7f0 2024-08-17T02:46:12.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.799+0000 7f6e3cc00640 1 -- 172.21.15.16:0/1393328581 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6e30073af0 con 0x7f6e4006d7f0 2024-08-17T02:46:12.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.802+0000 7f6e47600640 1 -- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6e1c065150 msgr2=0x7f6e1c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.802+0000 7f6e47600640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6e1c065150 0x7f6e1c067600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e40069430 tx=0x7f6e28008040 comp rx=0 tx=0).stop 2024-08-17T02:46:12.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.802+0000 7f6e47600640 1 -- 172.21.15.16:0/1393328581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4006d7f0 msgr2=0x7f6e400adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.802+0000 7f6e47600640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4006d7f0 0x7f6e400adb70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6e30035040 tx=0x7f6e30032000 comp rx=0 tx=0).stop 2024-08-17T02:46:12.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.802+0000 7f6e47600640 1 -- 172.21.15.16:0/1393328581 shutdown_connections 2024-08-17T02:46:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.802+0000 7f6e47600640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6e1c065150 0x7f6e1c067600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.802+0000 7f6e47600640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6e40155df0 0x7f6e4013db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.802+0000 7f6e47600640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e40155a20 0x7f6e40169fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.802+0000 7f6e47600640 1 --2- 172.21.15.16:0/1393328581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4006d7f0 0x7f6e400adb70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.802+0000 7f6e47600640 1 -- 172.21.15.16:0/1393328581 >> 172.21.15.16:0/1393328581 conn(0x7f6e400cb9b0 msgr2=0x7f6e40154430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.802+0000 7f6e47600640 1 -- 172.21.15.16:0/1393328581 shutdown_connections 2024-08-17T02:46:12.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.802+0000 7f6e47600640 1 -- 172.21.15.16:0/1393328581 wait complete. 2024-08-17T02:46:12.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:46:12.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.970+0000 7f35ea400640 1 Processor -- start 2024-08-17T02:46:12.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.970+0000 7f35ea400640 1 -- start start 2024-08-17T02:46:12.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35ea400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f35e40b1440 0x7f35e40caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35ea400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35e4140580 0x7f35e40c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35ea400640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e40caa80 con 0x7f35e41401b0 2024-08-17T02:46:12.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35ea400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e40cb450 con 0x7f35e4140580 2024-08-17T02:46:12.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35ea400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e40c9e70 con 0x7f35e40b1440 2024-08-17T02:46:12.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35e3e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35e4140580 0x7f35e40c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35e3400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f35e41401b0 0x7f35e40b0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57014/0 (socket says 172.21.15.16:57014) 2024-08-17T02:46:12.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35e3400640 1 -- 172.21.15.16:0/3595759556 learned_addr learned my addr 172.21.15.16:0/3595759556 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:12.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35e2a00640 1 --2- 172.21.15.16:0/3595759556 >> v2:172.21.15.142:3300/0 conn(0x7f35e40b1440 0x7f35e40caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35e3e00640 1 -- 172.21.15.16:0/3595759556 >> v2:172.21.15.142:3300/0 conn(0x7f35e40b1440 msgr2=0x7f35e40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:12.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35e3e00640 1 --2- 172.21.15.16:0/3595759556 >> v2:172.21.15.142:3300/0 conn(0x7f35e40b1440 0x7f35e40caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35e3e00640 1 -- 172.21.15.16:0/3595759556 >> v1:172.21.15.16:6789/0 conn(0x7f35e41401b0 legacy=0x7f35e40b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.972+0000 7f35e2000640 1 -- 172.21.15.16:0/3595759556 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986062583 0 0) 0x7f35e40caa80 con 0x7f35e41401b0 2024-08-17T02:46:12.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.973+0000 7f35e3e00640 1 -- 172.21.15.16:0/3595759556 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e40ca150 con 0x7f35e4140580 2024-08-17T02:46:12.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.974+0000 7f35e3e00640 1 --2- 172.21.15.16:0/3595759556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4140580 0x7f35e40c9040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f35e406aa80 tx=0x7f35dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7e1cd3513810a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:12.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.974+0000 7f35e2000640 1 -- 172.21.15.16:0/3595759556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc007550 con 0x7f35e4140580 2024-08-17T02:46:12.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.974+0000 7f35e2000640 1 -- 172.21.15.16:0/3595759556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc02d050 con 0x7f35e4140580 2024-08-17T02:46:12.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.974+0000 7f35e2000640 1 -- 172.21.15.16:0/3595759556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0078e0 con 0x7f35e4140580 2024-08-17T02:46:12.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.974+0000 7f35ea400640 1 -- 172.21.15.16:0/3595759556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4140580 msgr2=0x7f35e40c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.974+0000 7f35ea400640 1 --2- 172.21.15.16:0/3595759556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4140580 0x7f35e40c9040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f35e406aa80 tx=0x7f35dc009780 comp rx=0 tx=0).stop 2024-08-17T02:46:12.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.974+0000 7f35ea400640 1 -- 172.21.15.16:0/3595759556 shutdown_connections 2024-08-17T02:46:12.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.974+0000 7f35ea400640 1 --2- 172.21.15.16:0/3595759556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e4140580 0x7f35e40c9040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.974+0000 7f35ea400640 1 --2- 172.21.15.16:0/3595759556 >> v2:172.21.15.142:3300/0 conn(0x7f35e40b1440 0x7f35e40caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.974+0000 7f35ea400640 1 -- 172.21.15.16:0/3595759556 >> 172.21.15.16:0/3595759556 conn(0x7f35e40cb9b0 msgr2=0x7f35e40cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:12.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.975+0000 7f35ea400640 1 -- 172.21.15.16:0/3595759556 shutdown_connections 2024-08-17T02:46:12.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.975+0000 7f35ea400640 1 -- 172.21.15.16:0/3595759556 wait complete. 2024-08-17T02:46:12.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.976+0000 7f35ea400640 1 Processor -- start 2024-08-17T02:46:12.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.976+0000 7f35ea400640 1 -- start start 2024-08-17T02:46:12.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35ea400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35e40b1440 0x7f35e415b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35ea400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e41401b0 0x7f35e40a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35ea400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4140580 0x7f35e40a8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35ea400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40a7410 con 0x7f35e41401b0 2024-08-17T02:46:12.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35ea400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e41696a0 con 0x7f35e4140580 2024-08-17T02:46:12.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35ea400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4169980 con 0x7f35e40b1440 2024-08-17T02:46:12.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e41401b0 0x7f35e40a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35e2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e41401b0 0x7f35e40a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48536/0 (socket says 172.21.15.16:48536) 2024-08-17T02:46:12.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35e2a00640 1 -- 172.21.15.16:0/1246471794 learned_addr learned my addr 172.21.15.16:0/1246471794 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:12.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35e2a00640 1 -- 172.21.15.16:0/1246471794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35e40b1440 msgr2=0x7f35e415b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35e2a00640 1 --2- 172.21.15.16:0/1246471794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35e40b1440 0x7f35e415b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35e2a00640 1 -- 172.21.15.16:0/1246471794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4140580 msgr2=0x7f35e40a8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:12.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.977+0000 7f35e2a00640 1 --2- 172.21.15.16:0/1246471794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4140580 0x7f35e40a8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:12.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.978+0000 7f35e2a00640 1 -- 172.21.15.16:0/1246471794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc009a20 con 0x7f35e41401b0 2024-08-17T02:46:12.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.978+0000 7f35e2a00640 1 --2- 172.21.15.16:0/1246471794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e41401b0 0x7f35e40a5e30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d890 tx=0x7f35d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:12.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.978+0000 7f35daa00640 1 -- 172.21.15.16:0/1246471794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000de20 con 0x7f35e41401b0 2024-08-17T02:46:12.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.978+0000 7f35daa00640 1 -- 172.21.15.16:0/1246471794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d00098b0 con 0x7f35e41401b0 2024-08-17T02:46:12.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.978+0000 7f35daa00640 1 -- 172.21.15.16:0/1246471794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0009c70 con 0x7f35e41401b0 2024-08-17T02:46:12.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.978+0000 7f35ea400640 1 -- 172.21.15.16:0/1246471794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e40a7d20 con 0x7f35e41401b0 2024-08-17T02:46:12.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.978+0000 7f35ea400640 1 -- 172.21.15.16:0/1246471794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e40a82e0 con 0x7f35e41401b0 2024-08-17T02:46:12.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.979+0000 7f35ea400640 1 -- 172.21.15.16:0/1246471794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e40a8500 con 0x7f35e41401b0 2024-08-17T02:46:12.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.980+0000 7f35daa00640 1 -- 172.21.15.16:0/1246471794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f35d0014620 con 0x7f35e41401b0 2024-08-17T02:46:12.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.981+0000 7f35daa00640 1 --2- 172.21.15.16:0/1246471794 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f35c4065150 0x7f35c4067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:12.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.981+0000 7f35daa00640 1 -- 172.21.15.16:0/1246471794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f35d0082990 con 0x7f35e41401b0 2024-08-17T02:46:12.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.982+0000 7f35e3400640 1 --2- 172.21.15.16:0/1246471794 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f35c4065150 0x7f35c4067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:12.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.982+0000 7f35e3400640 1 --2- 172.21.15.16:0/1246471794 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f35c4065150 0x7f35c4067600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f35d40011e0 tx=0x7f35d4007940 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:12.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:12.985+0000 7f35daa00640 1 -- 172.21.15.16:0/1246471794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f35d0050d00 con 0x7f35e41401b0 2024-08-17T02:46:13.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.166+0000 7f35ea400640 1 -- 172.21.15.16:0/1246471794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35e4064db0 con 0x7f35e41401b0 2024-08-17T02:46:13.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.167+0000 7f35daa00640 1 -- 172.21.15.16:0/1246471794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12134 (secure 0 0 0) 0x7f35d0018090 con 0x7f35e41401b0 2024-08-17T02:46:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:46:13.167 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":29,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:46:09.726407+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:46:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.169+0000 7f35ea400640 1 -- 172.21.15.16:0/1246471794 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f35c4065150 msgr2=0x7f35c4067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.169+0000 7f35ea400640 1 --2- 172.21.15.16:0/1246471794 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f35c4065150 0x7f35c4067600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f35d40011e0 tx=0x7f35d4007940 comp rx=0 tx=0).stop 2024-08-17T02:46:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.169+0000 7f35ea400640 1 -- 172.21.15.16:0/1246471794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e41401b0 msgr2=0x7f35e40a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.169+0000 7f35ea400640 1 --2- 172.21.15.16:0/1246471794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e41401b0 0x7f35e40a5e30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f35d000d890 tx=0x7f35d0008040 comp rx=0 tx=0).stop 2024-08-17T02:46:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.169+0000 7f35ea400640 1 -- 172.21.15.16:0/1246471794 shutdown_connections 2024-08-17T02:46:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.169+0000 7f35ea400640 1 --2- 172.21.15.16:0/1246471794 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f35c4065150 0x7f35c4067600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.169+0000 7f35ea400640 1 --2- 172.21.15.16:0/1246471794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f35e4140580 0x7f35e40a8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.169+0000 7f35ea400640 1 --2- 172.21.15.16:0/1246471794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35e41401b0 0x7f35e40a5e30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.169+0000 7f35ea400640 1 --2- 172.21.15.16:0/1246471794 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f35e40b1440 0x7f35e415b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.169+0000 7f35ea400640 1 -- 172.21.15.16:0/1246471794 >> 172.21.15.16:0/1246471794 conn(0x7f35e40cb9b0 msgr2=0x7f35e40c0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.169+0000 7f35ea400640 1 -- 172.21.15.16:0/1246471794 shutdown_connections 2024-08-17T02:46:13.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.169+0000 7f35ea400640 1 -- 172.21.15.16:0/1246471794 wait complete. 2024-08-17T02:46:13.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:45:14.291903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-08-17T02:46:13.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:46:13.234 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-08-17T02:46:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.339+0000 7fc646200640 1 Processor -- start 2024-08-17T02:46:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.340+0000 7fc646200640 1 -- start start 2024-08-17T02:46:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.341+0000 7fc646200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc64015a0e0 0x7fc64006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.341+0000 7fc646200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6400ab010 0x7fc6400ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.341+0000 7fc646200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6400af5f0 con 0x7fc64006df00 2024-08-17T02:46:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.341+0000 7fc646200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc640159600 con 0x7fc6400ab010 2024-08-17T02:46:13.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.341+0000 7fc646200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6401598e0 con 0x7fc64015a0e0 2024-08-17T02:46:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.341+0000 7fc645800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6400ab010 0x7fc6400ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.341+0000 7fc644e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc64006df00 0x7fc6400aa900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57018/0 (socket says 172.21.15.16:57018) 2024-08-17T02:46:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.341+0000 7fc645800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc6400ab010 0x7fc6400ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48538/0 (socket says 172.21.15.16:48538) 2024-08-17T02:46:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.341+0000 7fc644e00640 1 -- 172.21.15.16:0/3991011241 learned_addr learned my addr 172.21.15.16:0/3991011241 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:13.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.341+0000 7fc63fe00640 1 --2- 172.21.15.16:0/3991011241 >> v2:172.21.15.142:3300/0 conn(0x7fc64015a0e0 0x7fc64006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.342+0000 7fc645800640 1 -- 172.21.15.16:0/3991011241 >> v2:172.21.15.142:3300/0 conn(0x7fc64015a0e0 msgr2=0x7fc64006e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.342+0000 7fc645800640 1 --2- 172.21.15.16:0/3991011241 >> v2:172.21.15.142:3300/0 conn(0x7fc64015a0e0 0x7fc64006e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.342+0000 7fc645800640 1 -- 172.21.15.16:0/3991011241 >> v1:172.21.15.16:6789/0 conn(0x7fc64006df00 legacy=0x7fc6400aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.342+0000 7fc645800640 1 -- 172.21.15.16:0/3991011241 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc640159bc0 con 0x7fc6400ab010 2024-08-17T02:46:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.342+0000 7fc63f400640 1 -- 172.21.15.16:0/3991011241 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149058870 0 0) 0x7fc6400af5f0 con 0x7fc64006df00 2024-08-17T02:46:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.343+0000 7fc645800640 1 --2- 172.21.15.16:0/3991011241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6400ab010 0x7fc6400ae8c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc6400c8890 tx=0x7fc638009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29f48810427ac4a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.343+0000 7fc63f400640 1 -- 172.21.15.16:0/3991011241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638007550 con 0x7fc6400ab010 2024-08-17T02:46:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.343+0000 7fc63f400640 1 -- 172.21.15.16:0/3991011241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63802d050 con 0x7fc6400ab010 2024-08-17T02:46:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.343+0000 7fc63f400640 1 -- 172.21.15.16:0/3991011241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6380078e0 con 0x7fc6400ab010 2024-08-17T02:46:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.343+0000 7fc646200640 1 -- 172.21.15.16:0/3991011241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6400ab010 msgr2=0x7fc6400ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.343+0000 7fc646200640 1 --2- 172.21.15.16:0/3991011241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6400ab010 0x7fc6400ae8c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc6400c8890 tx=0x7fc638009780 comp rx=0 tx=0).stop 2024-08-17T02:46:13.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.343+0000 7fc646200640 1 -- 172.21.15.16:0/3991011241 shutdown_connections 2024-08-17T02:46:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.343+0000 7fc646200640 1 --2- 172.21.15.16:0/3991011241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6400ab010 0x7fc6400ae8c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.343+0000 7fc646200640 1 --2- 172.21.15.16:0/3991011241 >> v2:172.21.15.142:3300/0 conn(0x7fc64015a0e0 0x7fc64006e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.343+0000 7fc646200640 1 -- 172.21.15.16:0/3991011241 >> 172.21.15.16:0/3991011241 conn(0x7fc640067750 msgr2=0x7fc64006c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.344+0000 7fc646200640 1 -- 172.21.15.16:0/3991011241 shutdown_connections 2024-08-17T02:46:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.344+0000 7fc646200640 1 -- 172.21.15.16:0/3991011241 wait complete. 2024-08-17T02:46:13.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.345+0000 7fc646200640 1 Processor -- start 2024-08-17T02:46:13.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.345+0000 7fc646200640 1 -- start start 2024-08-17T02:46:13.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.346+0000 7fc646200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64006df00 0x7fc640152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.346+0000 7fc646200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6400ab010 0x7fc6400c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.346+0000 7fc646200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc64015a0e0 0x7fc64015b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.346+0000 7fc646200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64015d260 con 0x7fc64006df00 2024-08-17T02:46:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.346+0000 7fc646200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64015d4e0 con 0x7fc6400ab010 2024-08-17T02:46:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.346+0000 7fc646200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64015d7c0 con 0x7fc64015a0e0 2024-08-17T02:46:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.346+0000 7fc644e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64006df00 0x7fc640152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.346+0000 7fc644e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64006df00 0x7fc640152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48554/0 (socket says 172.21.15.16:48554) 2024-08-17T02:46:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.346+0000 7fc644e00640 1 -- 172.21.15.16:0/752837225 learned_addr learned my addr 172.21.15.16:0/752837225 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.346+0000 7fc645800640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc64015a0e0 0x7fc64015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.346+0000 7fc63fe00640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6400ab010 0x7fc6400c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.347+0000 7fc644e00640 1 -- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc64015a0e0 msgr2=0x7fc64015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.347+0000 7fc644e00640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc64015a0e0 0x7fc64015b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.347+0000 7fc644e00640 1 -- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6400ab010 msgr2=0x7fc6400c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.347+0000 7fc644e00640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6400ab010 0x7fc6400c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.347+0000 7fc644e00640 1 -- 172.21.15.16:0/752837225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc638009a20 con 0x7fc64006df00 2024-08-17T02:46:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.347+0000 7fc644e00640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64006df00 0x7fc640152aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc63400c5e0 tx=0x7fc634007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.347+0000 7fc633e00640 1 -- 172.21.15.16:0/752837225 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc634005aa0 con 0x7fc64006df00 2024-08-17T02:46:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.347+0000 7fc633e00640 1 -- 172.21.15.16:0/752837225 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc634005c00 con 0x7fc64006df00 2024-08-17T02:46:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.347+0000 7fc646200640 1 -- 172.21.15.16:0/752837225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64015daa0 con 0x7fc64006df00 2024-08-17T02:46:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.347+0000 7fc633e00640 1 -- 172.21.15.16:0/752837225 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340070f0 con 0x7fc64006df00 2024-08-17T02:46:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.347+0000 7fc646200640 1 -- 172.21.15.16:0/752837225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64015e060 con 0x7fc64006df00 2024-08-17T02:46:13.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.348+0000 7fc646200640 1 -- 172.21.15.16:0/752837225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64015e280 con 0x7fc64006df00 2024-08-17T02:46:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.349+0000 7fc633e00640 1 -- 172.21.15.16:0/752837225 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fc634007520 con 0x7fc64006df00 2024-08-17T02:46:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.350+0000 7fc633e00640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc620065180 0x7fc620067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.350+0000 7fc633e00640 1 -- 172.21.15.16:0/752837225 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc634081c50 con 0x7fc64006df00 2024-08-17T02:46:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.351+0000 7fc63fe00640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc620065180 0x7fc620067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:13.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.351+0000 7fc63fe00640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc620065180 0x7fc620067630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc6400b1740 tx=0x7fc62c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:13.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.354+0000 7fc633e00640 1 -- 172.21.15.16:0/752837225 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc634050040 con 0x7fc64006df00 2024-08-17T02:46:13.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.536+0000 7fc646200640 1 -- 172.21.15.16:0/752837225 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc640071050 con 0x7fc64006df00 2024-08-17T02:46:13.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.537+0000 7fc633e00640 1 -- 172.21.15.16:0/752837225 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc634053660 con 0x7fc64006df00 2024-08-17T02:46:13.537 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-17T02:46:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.539+0000 7fc646200640 1 -- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc620065180 msgr2=0x7fc620067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.539+0000 7fc646200640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc620065180 0x7fc620067630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc6400b1740 tx=0x7fc62c008040 comp rx=0 tx=0).stop 2024-08-17T02:46:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.539+0000 7fc646200640 1 -- 172.21.15.16:0/752837225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64006df00 msgr2=0x7fc640152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.539+0000 7fc646200640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64006df00 0x7fc640152aa0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc63400c5e0 tx=0x7fc634007d60 comp rx=0 tx=0).stop 2024-08-17T02:46:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.539+0000 7fc646200640 1 -- 172.21.15.16:0/752837225 shutdown_connections 2024-08-17T02:46:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.539+0000 7fc646200640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc620065180 0x7fc620067630 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.539+0000 7fc646200640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc64015a0e0 0x7fc64015b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.539+0000 7fc646200640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6400ab010 0x7fc6400c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.539+0000 7fc646200640 1 --2- 172.21.15.16:0/752837225 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc64006df00 0x7fc640152aa0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.539+0000 7fc646200640 1 -- 172.21.15.16:0/752837225 >> 172.21.15.16:0/752837225 conn(0x7fc640067750 msgr2=0x7fc6400adbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:13.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.539+0000 7fc646200640 1 -- 172.21.15.16:0/752837225 shutdown_connections 2024-08-17T02:46:13.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.539+0000 7fc646200640 1 -- 172.21.15.16:0/752837225 wait complete. 2024-08-17T02:46:13.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-08-17T02:46:13.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.706+0000 7feca1a00640 1 Processor -- start 2024-08-17T02:46:13.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.706+0000 7feca1a00640 1 -- start start 2024-08-17T02:46:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.707+0000 7feca1a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fec9c155df0 0x7fec9c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.707+0000 7feca1a00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c156670 con 0x7fec9c06df00 2024-08-17T02:46:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.707+0000 7feca1a00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c06e5e0 con 0x7fec9c155a20 2024-08-17T02:46:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.708+0000 7feca1a00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9c0b0350 con 0x7fec9c155df0 2024-08-17T02:46:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.708+0000 7fec9a000640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fec9c06df00 0x7fec9c156b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57020/0 (socket says 172.21.15.16:57020) 2024-08-17T02:46:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.708+0000 7fec9a000640 1 -- 172.21.15.16:0/1262442209 learned_addr learned my addr 172.21.15.16:0/1262442209 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:13.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.708+0000 7fec9b400640 1 --2- 172.21.15.16:0/1262442209 >> v2:172.21.15.142:3300/0 conn(0x7fec9c155df0 0x7fec9c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.708+0000 7fec99600640 1 -- 172.21.15.16:0/1262442209 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674712809 0 0) 0x7fec9c156670 con 0x7fec9c06df00 2024-08-17T02:46:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.708+0000 7fec99600640 1 -- 172.21.15.16:0/1262442209 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec800034e0 con 0x7fec9c06df00 2024-08-17T02:46:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.708+0000 7fec9b400640 1 -- 172.21.15.16:0/1262442209 >> v1:172.21.15.142:6789/0 conn(0x7fec9c155a20 legacy=0x7fec9c06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.708+0000 7fec99600640 1 -- 172.21.15.16:0/1262442209 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2151139684 0 0) 0x7fec800034e0 con 0x7fec9c06df00 2024-08-17T02:46:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.709+0000 7fec9b400640 1 -- 172.21.15.16:0/1262442209 >> v1:172.21.15.16:6789/0 conn(0x7fec9c06df00 legacy=0x7fec9c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.709+0000 7fec9b400640 1 -- 172.21.15.16:0/1262442209 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c06e8c0 con 0x7fec9c155df0 2024-08-17T02:46:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.710+0000 7fec9b400640 1 --2- 172.21.15.16:0/1262442209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c155df0 0x7fec9c0adf10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fec94009e60 tx=0x7fec94009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=beecb51d511d440f server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.710+0000 7fec99600640 1 -- 172.21.15.16:0/1262442209 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88001470 con 0x7fec9c155df0 2024-08-17T02:46:13.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.710+0000 7fec99600640 1 -- 172.21.15.16:0/1262442209 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9402d050 con 0x7fec9c155df0 2024-08-17T02:46:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.710+0000 7fec99600640 1 -- 172.21.15.16:0/1262442209 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94031630 con 0x7fec9c155df0 2024-08-17T02:46:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.710+0000 7feca1a00640 1 -- 172.21.15.16:0/1262442209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c155df0 msgr2=0x7fec9c0adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.710+0000 7feca1a00640 1 --2- 172.21.15.16:0/1262442209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c155df0 0x7fec9c0adf10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fec94009e60 tx=0x7fec94009780 comp rx=0 tx=0).stop 2024-08-17T02:46:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.710+0000 7feca1a00640 1 -- 172.21.15.16:0/1262442209 shutdown_connections 2024-08-17T02:46:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.711+0000 7feca1a00640 1 --2- 172.21.15.16:0/1262442209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c155df0 0x7fec9c0adf10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.711+0000 7feca1a00640 1 -- 172.21.15.16:0/1262442209 >> 172.21.15.16:0/1262442209 conn(0x7fec9c0cb9b0 msgr2=0x7fec9c0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.711+0000 7feca1a00640 1 -- 172.21.15.16:0/1262442209 shutdown_connections 2024-08-17T02:46:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.711+0000 7feca1a00640 1 -- 172.21.15.16:0/1262442209 wait complete. 2024-08-17T02:46:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.712+0000 7feca1a00640 1 Processor -- start 2024-08-17T02:46:13.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.712+0000 7feca1a00640 1 -- start start 2024-08-17T02:46:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.713+0000 7feca1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c06df00 0x7fec9c15c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.713+0000 7feca1a00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec9c155a20 0x7fec9c15c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:13.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.713+0000 7feca1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec9c155df0 0x7fec9c0a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.713+0000 7feca1a00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c13cec0 con 0x7fec9c155df0 2024-08-17T02:46:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.713+0000 7feca1a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0ad440 con 0x7fec9c06df00 2024-08-17T02:46:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.713+0000 7feca1a00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c0ad720 con 0x7fec9c155a20 2024-08-17T02:46:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.713+0000 7fec9b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec9c155df0 0x7fec9c0a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.713+0000 7fec9b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec9c155df0 0x7fec9c0a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48564/0 (socket says 172.21.15.16:48564) 2024-08-17T02:46:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.713+0000 7fec9b400640 1 -- 172.21.15.16:0/3854340634 learned_addr learned my addr 172.21.15.16:0/3854340634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.714+0000 7fec9a000640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec9c155a20 0x7fec9c15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.714+0000 7fec9aa00640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c06df00 0x7fec9c15c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.714+0000 7fec9b400640 1 -- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec9c155a20 msgr2=0x7fec9c15c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.714+0000 7fec9b400640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec9c155a20 0x7fec9c15c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.714+0000 7fec9b400640 1 -- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c06df00 msgr2=0x7fec9c15c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.714+0000 7fec9b400640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c06df00 0x7fec9c15c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.714+0000 7fec9b400640 1 -- 172.21.15.16:0/3854340634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec94009a20 con 0x7fec9c155df0 2024-08-17T02:46:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.714+0000 7fec9aa00640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c06df00 0x7fec9c15c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:13.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.714+0000 7fec9b400640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec9c155df0 0x7fec9c0a55f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec940099f0 tx=0x7fec94037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.714+0000 7fec92000640 1 -- 172.21.15.16:0/3854340634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9403ba70 con 0x7fec9c155df0 2024-08-17T02:46:13.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.715+0000 7fec92000640 1 -- 172.21.15.16:0/3854340634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec94040070 con 0x7fec9c155df0 2024-08-17T02:46:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.715+0000 7fec92000640 1 -- 172.21.15.16:0/3854340634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9403be40 con 0x7fec9c155df0 2024-08-17T02:46:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.715+0000 7feca1a00640 1 -- 172.21.15.16:0/3854340634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c0ada00 con 0x7fec9c155df0 2024-08-17T02:46:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.715+0000 7feca1a00640 1 -- 172.21.15.16:0/3854340634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c0a5b30 con 0x7fec9c155df0 2024-08-17T02:46:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.716+0000 7feca1a00640 1 -- 172.21.15.16:0/3854340634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9c13d070 con 0x7fec9c155df0 2024-08-17T02:46:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.717+0000 7fec92000640 1 -- 172.21.15.16:0/3854340634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fec94003020 con 0x7fec9c155df0 2024-08-17T02:46:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.717+0000 7fec92000640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fec7c065180 0x7fec7c067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:13.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.717+0000 7fec92000640 1 -- 172.21.15.16:0/3854340634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fec94042070 con 0x7fec9c155df0 2024-08-17T02:46:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.718+0000 7fec9aa00640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fec7c065180 0x7fec7c067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:13.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.719+0000 7fec9aa00640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fec7c065180 0x7fec7c067630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fec8c001820 tx=0x7fec8c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:13.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.721+0000 7fec92000640 1 -- 172.21.15.16:0/3854340634 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fec9c13d070 con 0x7fec9c155df0 2024-08-17T02:46:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.906+0000 7feca1a00640 1 -- 172.21.15.16:0/3854340634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fec9c13d070 con 0x7fec9c155df0 2024-08-17T02:46:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.907+0000 7fec92000640 1 -- 172.21.15.16:0/3854340634 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7fec9404c060 con 0x7fec9c155df0 2024-08-17T02:46:13.907 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:46:13.907 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-08-17T02:46:13.907 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:13.907 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-08-17T02:46:13.907 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-08-17T02:46:13.908 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:13.908 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-08-17T02:46:13.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.909+0000 7feca1a00640 1 -- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fec7c065180 msgr2=0x7fec7c067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.909+0000 7feca1a00640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fec7c065180 0x7fec7c067630 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fec8c001820 tx=0x7fec8c009040 comp rx=0 tx=0).stop 2024-08-17T02:46:13.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.909+0000 7feca1a00640 1 -- 172.21.15.16:0/3854340634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec9c155df0 msgr2=0x7fec9c0a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.909+0000 7feca1a00640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec9c155df0 0x7fec9c0a55f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fec940099f0 tx=0x7fec94037000 comp rx=0 tx=0).stop 2024-08-17T02:46:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.909+0000 7feca1a00640 1 -- 172.21.15.16:0/3854340634 shutdown_connections 2024-08-17T02:46:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.909+0000 7feca1a00640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fec7c065180 0x7fec7c067630 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.909+0000 7feca1a00640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec9c155df0 0x7fec9c0a55f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.909+0000 7feca1a00640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fec9c155a20 0x7fec9c15c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.909+0000 7feca1a00640 1 --2- 172.21.15.16:0/3854340634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec9c06df00 0x7fec9c15c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.909+0000 7feca1a00640 1 -- 172.21.15.16:0/3854340634 >> 172.21.15.16:0/3854340634 conn(0x7fec9c0cb9b0 msgr2=0x7fec9c154c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.909+0000 7feca1a00640 1 -- 172.21.15.16:0/3854340634 shutdown_connections 2024-08-17T02:46:13.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:13.909+0000 7feca1a00640 1 -- 172.21.15.16:0/3854340634 wait complete. 2024-08-17T02:46:13.926 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-17T02:46:13.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-17T02:46:14.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.081+0000 7f7237e00640 1 Processor -- start 2024-08-17T02:46:14.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.081+0000 7f7237e00640 1 -- start start 2024-08-17T02:46:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.083+0000 7f7237e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f723006d9e0 0x7f723006ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.083+0000 7f7237e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f72300cb4f0 0x7f72300acc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.083+0000 7f7237e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72300c83e0 con 0x7f72300cb0c0 2024-08-17T02:46:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.083+0000 7f7237e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72300aed90 con 0x7f72300cb4f0 2024-08-17T02:46:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.083+0000 7f7237e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72300af070 con 0x7f723006d9e0 2024-08-17T02:46:14.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.083+0000 7f7235e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f72300cb4f0 0x7f72300acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:14.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.083+0000 7f7235400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f723006d9e0 0x7f723006ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.083+0000 7f7235e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f72300cb4f0 0x7f72300acc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:53504/0 (socket says 172.21.15.16:53504) 2024-08-17T02:46:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.083+0000 7f7235e00640 1 -- 172.21.15.16:0/661744948 learned_addr learned my addr 172.21.15.16:0/661744948 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.083+0000 7f7234a00640 1 --1- 172.21.15.16:0/661744948 >> v1:172.21.15.142:6789/0 conn(0x7f72300cb0c0 0x7f7230154d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:58980/0 (socket says 172.21.15.16:58980) 2024-08-17T02:46:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.084+0000 7f7235400640 1 -- 172.21.15.16:0/661744948 >> v2:172.21.15.142:3300/0 conn(0x7f72300cb4f0 msgr2=0x7f72300acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.084+0000 7f7235400640 1 --2- 172.21.15.16:0/661744948 >> v2:172.21.15.142:3300/0 conn(0x7f72300cb4f0 0x7f72300acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.084+0000 7f7235400640 1 -- 172.21.15.16:0/661744948 >> v1:172.21.15.142:6789/0 conn(0x7f72300cb0c0 legacy=0x7f7230154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.084+0000 7f7235400640 1 -- 172.21.15.16:0/661744948 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72300af350 con 0x7f723006d9e0 2024-08-17T02:46:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.084+0000 7f722fe00640 1 -- 172.21.15.16:0/661744948 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831847157 0 0) 0x7f72300c83e0 con 0x7f72300cb0c0 2024-08-17T02:46:14.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.084+0000 7f7235e00640 1 --2- 172.21.15.16:0/661744948 >> v2:172.21.15.142:3300/0 conn(0x7f72300cb4f0 0x7f72300acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:46:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.085+0000 7f7235400640 1 --2- 172.21.15.16:0/661744948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f723006d9e0 0x7f723006ddb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7224009850 tx=0x7f72240090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee489ab008288d6e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:14.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.085+0000 7f722fe00640 1 -- 172.21.15.16:0/661744948 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722402ac80 con 0x7f723006d9e0 2024-08-17T02:46:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.085+0000 7f722fe00640 1 -- 172.21.15.16:0/661744948 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722402ade0 con 0x7f723006d9e0 2024-08-17T02:46:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.085+0000 7f722fe00640 1 -- 172.21.15.16:0/661744948 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224032670 con 0x7f723006d9e0 2024-08-17T02:46:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.086+0000 7f7237e00640 1 -- 172.21.15.16:0/661744948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f723006d9e0 msgr2=0x7f723006ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.086+0000 7f7237e00640 1 --2- 172.21.15.16:0/661744948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f723006d9e0 0x7f723006ddb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7224009850 tx=0x7f72240090f0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.086+0000 7f7237e00640 1 -- 172.21.15.16:0/661744948 shutdown_connections 2024-08-17T02:46:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.086+0000 7f7237e00640 1 --2- 172.21.15.16:0/661744948 >> v2:172.21.15.142:3300/0 conn(0x7f72300cb4f0 0x7f72300acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.086+0000 7f7237e00640 1 --2- 172.21.15.16:0/661744948 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f723006d9e0 0x7f723006ddb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.086+0000 7f7237e00640 1 -- 172.21.15.16:0/661744948 >> 172.21.15.16:0/661744948 conn(0x7f7230067750 msgr2=0x7f72300cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.086+0000 7f7237e00640 1 -- 172.21.15.16:0/661744948 shutdown_connections 2024-08-17T02:46:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.086+0000 7f7237e00640 1 -- 172.21.15.16:0/661744948 wait complete. 2024-08-17T02:46:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.088+0000 7f7237e00640 1 Processor -- start 2024-08-17T02:46:14.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.088+0000 7f7237e00640 1 -- start start 2024-08-17T02:46:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7237e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f723006d9e0 0x7f72300abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7237e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72300cb0c0 0x7f7230169970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7237e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72300cb4f0 0x7f72300c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7237e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72301521c0 con 0x7f723006d9e0 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7237e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7230169eb0 con 0x7f72300cb4f0 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7237e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7230151640 con 0x7f72300cb0c0 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7235400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f723006d9e0 0x7f72300abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7235400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f723006d9e0 0x7f72300abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42972/0 (socket says 172.21.15.16:42972) 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7235400640 1 -- 172.21.15.16:0/141736065 learned_addr learned my addr 172.21.15.16:0/141736065 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7234a00640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72300cb0c0 0x7f7230169970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7235e00640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72300cb4f0 0x7f72300c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7235400640 1 -- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72300cb0c0 msgr2=0x7f7230169970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7235400640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72300cb0c0 0x7f7230169970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7235400640 1 -- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72300cb4f0 msgr2=0x7f72300c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7235400640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72300cb4f0 0x7f72300c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7235400640 1 -- 172.21.15.16:0/141736065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722800bcf0 con 0x7f723006d9e0 2024-08-17T02:46:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.089+0000 7f7234a00640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72300cb0c0 0x7f7230169970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.090+0000 7f7235400640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f723006d9e0 0x7f72300abf90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7224009850 tx=0x7f7224032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.090+0000 7f722cc00640 1 -- 172.21.15.16:0/141736065 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722403d700 con 0x7f723006d9e0 2024-08-17T02:46:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.090+0000 7f722cc00640 1 -- 172.21.15.16:0/141736065 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722403d860 con 0x7f723006d9e0 2024-08-17T02:46:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.090+0000 7f722cc00640 1 -- 172.21.15.16:0/141736065 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722403dc40 con 0x7f723006d9e0 2024-08-17T02:46:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.090+0000 7f7237e00640 1 -- 172.21.15.16:0/141736065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7224009390 con 0x7f723006d9e0 2024-08-17T02:46:14.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.090+0000 7f7237e00640 1 -- 172.21.15.16:0/141736065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7230151c00 con 0x7f723006d9e0 2024-08-17T02:46:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.091+0000 7f7237e00640 1 -- 172.21.15.16:0/141736065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7230064db0 con 0x7f723006d9e0 2024-08-17T02:46:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.092+0000 7f722cc00640 1 -- 172.21.15.16:0/141736065 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f7224039040 con 0x7f723006d9e0 2024-08-17T02:46:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.093+0000 7f722cc00640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7210065150 0x7f7210067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.093+0000 7f722cc00640 1 -- 172.21.15.16:0/141736065 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f72240a1cd0 con 0x7f723006d9e0 2024-08-17T02:46:14.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.093+0000 7f7234a00640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7210065150 0x7f7210067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:14.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.094+0000 7f7234a00640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7210065150 0x7f7210067600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7230153750 tx=0x7f721c007c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:14.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.097+0000 7f722cc00640 1 -- 172.21.15.16:0/141736065 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7224070040 con 0x7f723006d9e0 2024-08-17T02:46:14.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.279+0000 7f7237e00640 1 -- 172.21.15.16:0/141736065 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f723006e970 con 0x7f723006d9e0 2024-08-17T02:46:14.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.750+0000 7f722cc00640 1 -- 172.21.15.16:0/141736065 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7224073660 con 0x7f723006d9e0 2024-08-17T02:46:14.751 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-08-17T02:46:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.753+0000 7f7237e00640 1 -- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7210065150 msgr2=0x7f7210067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.753+0000 7f7237e00640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7210065150 0x7f7210067600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7230153750 tx=0x7f721c007c00 comp rx=0 tx=0).stop 2024-08-17T02:46:14.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.753+0000 7f7237e00640 1 -- 172.21.15.16:0/141736065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f723006d9e0 msgr2=0x7f72300abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:14.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.753+0000 7f7237e00640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f723006d9e0 0x7f72300abf90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7224009850 tx=0x7f7224032000 comp rx=0 tx=0).stop 2024-08-17T02:46:14.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.753+0000 7f7237e00640 1 -- 172.21.15.16:0/141736065 shutdown_connections 2024-08-17T02:46:14.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.753+0000 7f7237e00640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7210065150 0x7f7210067600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.753+0000 7f7237e00640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72300cb4f0 0x7f72300c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.753+0000 7f7237e00640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72300cb0c0 0x7f7230169970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.753+0000 7f7237e00640 1 --2- 172.21.15.16:0/141736065 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f723006d9e0 0x7f72300abf90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.753+0000 7f7237e00640 1 -- 172.21.15.16:0/141736065 >> 172.21.15.16:0/141736065 conn(0x7f7230067750 msgr2=0x7f7230159530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:14.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.753+0000 7f7237e00640 1 -- 172.21.15.16:0/141736065 shutdown_connections 2024-08-17T02:46:14.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.753+0000 7f7237e00640 1 -- 172.21.15.16:0/141736065 wait complete. 2024-08-17T02:46:14.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-17T02:46:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.933+0000 7fb411c00640 1 Processor -- start 2024-08-17T02:46:14.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.933+0000 7fb411c00640 1 -- start start 2024-08-17T02:46:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.934+0000 7fb411c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb40c156890 0x7fb40c06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.934+0000 7fb411c00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb40c156c60 0x7fb40c06e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.934+0000 7fb411c00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40c0ae9b0 con 0x7fb40c0c8130 2024-08-17T02:46:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.934+0000 7fb411c00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40c0aa730 con 0x7fb40c156c60 2024-08-17T02:46:14.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.934+0000 7fb411c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40c0aa9f0 con 0x7fb40c156890 2024-08-17T02:46:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.935+0000 7fb403e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb40c156c60 0x7fb40c06e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.935+0000 7fb40b400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb40c0c8130 0x7fb40c0c8560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57208/0 (socket says 172.21.15.16:57208) 2024-08-17T02:46:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.935+0000 7fb40b400640 1 -- 172.21.15.16:0/121291200 learned_addr learned my addr 172.21.15.16:0/121291200 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.935+0000 7fb403e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb40c156c60 0x7fb40c06e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42978/0 (socket says 172.21.15.16:42978) 2024-08-17T02:46:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.935+0000 7fb40aa00640 1 --2- 172.21.15.16:0/121291200 >> v2:172.21.15.142:3300/0 conn(0x7fb40c156890 0x7fb40c06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:14.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.935+0000 7fb403e00640 1 -- 172.21.15.16:0/121291200 >> v2:172.21.15.142:3300/0 conn(0x7fb40c156890 msgr2=0x7fb40c06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.935+0000 7fb403e00640 1 --2- 172.21.15.16:0/121291200 >> v2:172.21.15.142:3300/0 conn(0x7fb40c156890 0x7fb40c06dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.935+0000 7fb403e00640 1 -- 172.21.15.16:0/121291200 >> v1:172.21.15.16:6789/0 conn(0x7fb40c0c8130 legacy=0x7fb40c0c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.935+0000 7fb403e00640 1 -- 172.21.15.16:0/121291200 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c0ab320 con 0x7fb40c156c60 2024-08-17T02:46:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.935+0000 7fb40a000640 1 -- 172.21.15.16:0/121291200 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087437294 0 0) 0x7fb40c0ae9b0 con 0x7fb40c0c8130 2024-08-17T02:46:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.936+0000 7fb403e00640 1 --2- 172.21.15.16:0/121291200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c156c60 0x7fb40c06e420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb40c06c5c0 tx=0x7fb3f40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1f42368bec6a4b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:14.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.937+0000 7fb40a000640 1 -- 172.21.15.16:0/121291200 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f402ad20 con 0x7fb40c156c60 2024-08-17T02:46:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.937+0000 7fb40a000640 1 -- 172.21.15.16:0/121291200 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f402ae80 con 0x7fb40c156c60 2024-08-17T02:46:14.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.937+0000 7fb40a000640 1 -- 172.21.15.16:0/121291200 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4032670 con 0x7fb40c156c60 2024-08-17T02:46:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.937+0000 7fb411c00640 1 -- 172.21.15.16:0/121291200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c156c60 msgr2=0x7fb40c06e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.937+0000 7fb411c00640 1 --2- 172.21.15.16:0/121291200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c156c60 0x7fb40c06e420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb40c06c5c0 tx=0x7fb3f40090f0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.937+0000 7fb411c00640 1 -- 172.21.15.16:0/121291200 shutdown_connections 2024-08-17T02:46:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.937+0000 7fb411c00640 1 --2- 172.21.15.16:0/121291200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c156c60 0x7fb40c06e420 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.937+0000 7fb411c00640 1 --2- 172.21.15.16:0/121291200 >> v2:172.21.15.142:3300/0 conn(0x7fb40c156890 0x7fb40c06dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.937+0000 7fb411c00640 1 -- 172.21.15.16:0/121291200 >> 172.21.15.16:0/121291200 conn(0x7fb40c067750 msgr2=0x7fb40c06c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.937+0000 7fb411c00640 1 -- 172.21.15.16:0/121291200 shutdown_connections 2024-08-17T02:46:14.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.937+0000 7fb411c00640 1 -- 172.21.15.16:0/121291200 wait complete. 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.939+0000 7fb411c00640 1 Processor -- start 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.939+0000 7fb411c00640 1 -- start start 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb411c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c0c8130 0x7fb40c14acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb411c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c156890 0x7fb40c0acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb411c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb40c156c60 0x7fb40c0ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb411c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c16a170 con 0x7fb40c156890 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb411c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c169560 con 0x7fb40c0c8130 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb411c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c169840 con 0x7fb40c156c60 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb403e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c156890 0x7fb40c0acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb403e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c156890 0x7fb40c0acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42990/0 (socket says 172.21.15.16:42990) 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb403e00640 1 -- 172.21.15.16:0/928900559 learned_addr learned my addr 172.21.15.16:0/928900559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb40aa00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c0c8130 0x7fb40c14acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb40b400640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb40c156c60 0x7fb40c0ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:14.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb403e00640 1 -- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb40c156c60 msgr2=0x7fb40c0ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb403e00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb40c156c60 0x7fb40c0ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb403e00640 1 -- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c0c8130 msgr2=0x7fb40c14acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb403e00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c0c8130 0x7fb40c14acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb403e00640 1 -- 172.21.15.16:0/928900559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f4009390 con 0x7fb40c156890 2024-08-17T02:46:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb40aa00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c0c8130 0x7fb40c14acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:46:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.940+0000 7fb40b400640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb40c156c60 0x7fb40c0ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:46:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.941+0000 7fb403e00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c156890 0x7fb40c0acb40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4035040 tx=0x7fb3f4032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.941+0000 7fb402000640 1 -- 172.21.15.16:0/928900559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4031700 con 0x7fb40c156890 2024-08-17T02:46:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.941+0000 7fb402000640 1 -- 172.21.15.16:0/928900559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f4031860 con 0x7fb40c156890 2024-08-17T02:46:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.941+0000 7fb402000640 1 -- 172.21.15.16:0/928900559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4031c00 con 0x7fb40c156890 2024-08-17T02:46:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.941+0000 7fb411c00640 1 -- 172.21.15.16:0/928900559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40c169b20 con 0x7fb40c156890 2024-08-17T02:46:14.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.941+0000 7fb411c00640 1 -- 172.21.15.16:0/928900559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40c14b230 con 0x7fb40c156890 2024-08-17T02:46:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.942+0000 7fb411c00640 1 -- 172.21.15.16:0/928900559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40c0aee00 con 0x7fb40c156890 2024-08-17T02:46:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.943+0000 7fb402000640 1 -- 172.21.15.16:0/928900559 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fb3f4039040 con 0x7fb40c156890 2024-08-17T02:46:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.944+0000 7fb402000640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb3e8065150 0x7fb3e8067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.944+0000 7fb402000640 1 -- 172.21.15.16:0/928900559 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fb3f40a2490 con 0x7fb40c156890 2024-08-17T02:46:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.944+0000 7fb40aa00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb3e8065150 0x7fb3e8067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:14.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.945+0000 7fb40aa00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb3e8065150 0x7fb3e8067600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc006e70 tx=0x7fb3fc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:14.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:14.948+0000 7fb402000640 1 -- 172.21.15.16:0/928900559 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb40c0aee00 con 0x7fb40c156890 2024-08-17T02:46:15.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.130+0000 7fb411c00640 1 -- 172.21.15.16:0/928900559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb40c0aee00 con 0x7fb40c156890 2024-08-17T02:46:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.760+0000 7fb402000640 1 -- 172.21.15.16:0/928900559 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb40c0aee00 con 0x7fb40c156890 2024-08-17T02:46:15.761 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-08-17T02:46:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.768+0000 7fb411c00640 1 -- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb3e8065150 msgr2=0x7fb3e8067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.768+0000 7fb411c00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb3e8065150 0x7fb3e8067600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc006e70 tx=0x7fb3fc008040 comp rx=0 tx=0).stop 2024-08-17T02:46:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.768+0000 7fb411c00640 1 -- 172.21.15.16:0/928900559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c156890 msgr2=0x7fb40c0acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.768+0000 7fb411c00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c156890 0x7fb40c0acb40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4035040 tx=0x7fb3f4032000 comp rx=0 tx=0).stop 2024-08-17T02:46:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.769+0000 7fb411c00640 1 -- 172.21.15.16:0/928900559 shutdown_connections 2024-08-17T02:46:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.769+0000 7fb411c00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb3e8065150 0x7fb3e8067600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.769+0000 7fb411c00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb40c156c60 0x7fb40c0ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.769+0000 7fb411c00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb40c156890 0x7fb40c0acb40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.769+0000 7fb411c00640 1 --2- 172.21.15.16:0/928900559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb40c0c8130 0x7fb40c14acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.769+0000 7fb411c00640 1 -- 172.21.15.16:0/928900559 >> 172.21.15.16:0/928900559 conn(0x7fb40c067750 msgr2=0x7fb40c068b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.770+0000 7fb411c00640 1 -- 172.21.15.16:0/928900559 shutdown_connections 2024-08-17T02:46:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.771+0000 7fb411c00640 1 -- 172.21.15.16:0/928900559 wait complete. 2024-08-17T02:46:15.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-17T02:46:15.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.954+0000 7f07a3e00640 1 Processor -- start 2024-08-17T02:46:15.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.955+0000 7f07a3e00640 1 -- start start 2024-08-17T02:46:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.956+0000 7f07a3e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f07a4158940 0x7f07a415d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.956+0000 7f07a3e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07a4158d10 0x7f07a4066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.956+0000 7f07a3e00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a406e660 con 0x7f07a4159190 2024-08-17T02:46:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.956+0000 7f07a3e00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a40a9b70 con 0x7f07a4158d10 2024-08-17T02:46:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.956+0000 7f07a3e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a406da50 con 0x7f07a4158940 2024-08-17T02:46:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.956+0000 7f079be00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07a4158d10 0x7f07a4066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.956+0000 7f079be00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07a4158d10 0x7f07a4066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42996/0 (socket says 172.21.15.16:42996) 2024-08-17T02:46:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.956+0000 7f079be00640 1 -- 172.21.15.16:0/95682006 learned_addr learned my addr 172.21.15.16:0/95682006 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.956+0000 7f07a3400640 1 --1- 172.21.15.16:0/95682006 >> v1:172.21.15.16:6789/0 conn(0x7f07a4159190 0x7f07a40af8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57218/0 (socket says 172.21.15.16:57218) 2024-08-17T02:46:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.956+0000 7f07a2a00640 1 --2- 172.21.15.16:0/95682006 >> v2:172.21.15.142:3300/0 conn(0x7f07a4158940 0x7f07a415d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:15.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.957+0000 7f079be00640 1 -- 172.21.15.16:0/95682006 >> v2:172.21.15.142:3300/0 conn(0x7f07a4158940 msgr2=0x7f07a415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:15.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.957+0000 7f079be00640 1 --2- 172.21.15.16:0/95682006 >> v2:172.21.15.142:3300/0 conn(0x7f07a4158940 0x7f07a415d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:15.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.957+0000 7f079be00640 1 -- 172.21.15.16:0/95682006 >> v1:172.21.15.16:6789/0 conn(0x7f07a4159190 legacy=0x7f07a40af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:15.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.957+0000 7f079be00640 1 -- 172.21.15.16:0/95682006 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a406dd30 con 0x7f07a4158d10 2024-08-17T02:46:15.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.957+0000 7f07a2000640 1 -- 172.21.15.16:0/95682006 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081716573 0 0) 0x7f07a406e660 con 0x7f07a4159190 2024-08-17T02:46:15.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.958+0000 7f079be00640 1 --2- 172.21.15.16:0/95682006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4158d10 0x7f07a4066cd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f07a4064440 tx=0x7f078c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b837becdc73f7a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:15.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.958+0000 7f07a2000640 1 -- 172.21.15.16:0/95682006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c02ad40 con 0x7f07a4158d10 2024-08-17T02:46:15.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.958+0000 7f07a2000640 1 -- 172.21.15.16:0/95682006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078c032440 con 0x7f07a4158d10 2024-08-17T02:46:15.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.958+0000 7f07a2000640 1 -- 172.21.15.16:0/95682006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078c02aea0 con 0x7f07a4158d10 2024-08-17T02:46:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.958+0000 7f07a3e00640 1 -- 172.21.15.16:0/95682006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4158d10 msgr2=0x7f07a4066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.958+0000 7f07a3e00640 1 --2- 172.21.15.16:0/95682006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4158d10 0x7f07a4066cd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f07a4064440 tx=0x7f078c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.959+0000 7f07a3e00640 1 -- 172.21.15.16:0/95682006 shutdown_connections 2024-08-17T02:46:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.959+0000 7f07a3e00640 1 --2- 172.21.15.16:0/95682006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4158d10 0x7f07a4066cd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.959+0000 7f07a3e00640 1 --2- 172.21.15.16:0/95682006 >> v2:172.21.15.142:3300/0 conn(0x7f07a4158940 0x7f07a415d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.959+0000 7f07a3e00640 1 -- 172.21.15.16:0/95682006 >> 172.21.15.16:0/95682006 conn(0x7f07a4067de0 msgr2=0x7f07a40681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.959+0000 7f07a3e00640 1 -- 172.21.15.16:0/95682006 shutdown_connections 2024-08-17T02:46:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.959+0000 7f07a3e00640 1 -- 172.21.15.16:0/95682006 wait complete. 2024-08-17T02:46:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.960+0000 7f07a3e00640 1 Processor -- start 2024-08-17T02:46:15.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.960+0000 7f07a3e00640 1 -- start start 2024-08-17T02:46:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.961+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4158940 0x7f07a415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.961+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07a4158d10 0x7f07a415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.961+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a4159190 0x7f07a413fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.961+0000 7f07a3e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a415ab90 con 0x7f07a4158940 2024-08-17T02:46:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.961+0000 7f07a3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a415b9b0 con 0x7f07a4159190 2024-08-17T02:46:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.961+0000 7f07a3e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a415d250 con 0x7f07a4158d10 2024-08-17T02:46:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.961+0000 7f07a2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4158940 0x7f07a415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.961+0000 7f07a2a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4158940 0x7f07a415a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43004/0 (socket says 172.21.15.16:43004) 2024-08-17T02:46:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.962+0000 7f07a2a00640 1 -- 172.21.15.16:0/2467518666 learned_addr learned my addr 172.21.15.16:0/2467518666 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.962+0000 7f07a3400640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a4159190 0x7f07a413fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:15.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.962+0000 7f079be00640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07a4158d10 0x7f07a415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.962+0000 7f07a2a00640 1 -- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07a4158d10 msgr2=0x7f07a415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.962+0000 7f07a2a00640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07a4158d10 0x7f07a415cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.962+0000 7f07a2a00640 1 -- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a4159190 msgr2=0x7f07a413fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.962+0000 7f07a2a00640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a4159190 0x7f07a413fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.962+0000 7f07a2a00640 1 -- 172.21.15.16:0/2467518666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c009450 con 0x7f07a4158940 2024-08-17T02:46:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.962+0000 7f07a2a00640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4158940 0x7f07a415a650 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f079400e930 tx=0x7f07940090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.962+0000 7f079a000640 1 -- 172.21.15.16:0/2467518666 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079400c8e0 con 0x7f07a4158940 2024-08-17T02:46:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.963+0000 7f079a000640 1 -- 172.21.15.16:0/2467518666 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079400ca40 con 0x7f07a4158940 2024-08-17T02:46:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.963+0000 7f07a3e00640 1 -- 172.21.15.16:0/2467518666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a415ada0 con 0x7f07a4158940 2024-08-17T02:46:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.963+0000 7f079a000640 1 -- 172.21.15.16:0/2467518666 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079400cde0 con 0x7f07a4158940 2024-08-17T02:46:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.963+0000 7f07a3e00640 1 -- 172.21.15.16:0/2467518666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a415b360 con 0x7f07a4158940 2024-08-17T02:46:15.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.964+0000 7f07a3e00640 1 -- 172.21.15.16:0/2467518666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a4071050 con 0x7f07a4158940 2024-08-17T02:46:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.965+0000 7f079a000640 1 -- 172.21.15.16:0/2467518666 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f0794013680 con 0x7f07a4158940 2024-08-17T02:46:15.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.965+0000 7f079a000640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0780065180 0x7f0780067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.966+0000 7f079a000640 1 -- 172.21.15.16:0/2467518666 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f0794082550 con 0x7f07a4158940 2024-08-17T02:46:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.966+0000 7f079be00640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0780065180 0x7f0780067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:15.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.967+0000 7f079be00640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0780065180 0x7f0780067630 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f078c034000 tx=0x7f078c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:15.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:15.970+0000 7f079a000640 1 -- 172.21.15.16:0/2467518666 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0794050680 con 0x7f07a4158940 2024-08-17T02:46:16.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.189+0000 7f07a3e00640 1 -- 172.21.15.16:0/2467518666 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f07a40aa8c0 con 0x7f07a4158940 2024-08-17T02:46:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.773+0000 7f079a000640 1 -- 172.21.15.16:0/2467518666 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0794053ca0 con 0x7f07a4158940 2024-08-17T02:46:16.775 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 4 and data pool 5 2024-08-17T02:46:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.781+0000 7f07a3e00640 1 -- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0780065180 msgr2=0x7f0780067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.781+0000 7f07a3e00640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0780065180 0x7f0780067630 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f078c034000 tx=0x7f078c032000 comp rx=0 tx=0).stop 2024-08-17T02:46:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.781+0000 7f07a3e00640 1 -- 172.21.15.16:0/2467518666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4158940 msgr2=0x7f07a415a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.781+0000 7f07a3e00640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4158940 0x7f07a415a650 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f079400e930 tx=0x7f07940090f0 comp rx=0 tx=0).stop 2024-08-17T02:46:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.781+0000 7f07a3e00640 1 -- 172.21.15.16:0/2467518666 shutdown_connections 2024-08-17T02:46:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.781+0000 7f07a3e00640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0780065180 0x7f0780067630 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.781+0000 7f07a3e00640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07a4159190 0x7f07a413fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.781+0000 7f07a3e00640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f07a4158d10 0x7f07a415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.781+0000 7f07a3e00640 1 --2- 172.21.15.16:0/2467518666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07a4158940 0x7f07a415a650 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.781+0000 7f07a3e00640 1 -- 172.21.15.16:0/2467518666 >> 172.21.15.16:0/2467518666 conn(0x7f07a4067de0 msgr2=0x7f07a4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.783+0000 7f07a3e00640 1 -- 172.21.15.16:0/2467518666 shutdown_connections 2024-08-17T02:46:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.783+0000 7f07a3e00640 1 -- 172.21.15.16:0/2467518666 wait complete. 2024-08-17T02:46:16.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:46:16.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.970+0000 7fce4be00640 1 Processor -- start 2024-08-17T02:46:16.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.971+0000 7fce4be00640 1 -- start start 2024-08-17T02:46:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.972+0000 7fce4be00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fce4c069710 0x7fce4c0ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.972+0000 7fce4be00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c0aec20 con 0x7fce4c158bd0 2024-08-17T02:46:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.972+0000 7fce4be00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c0ae330 con 0x7fce4c0ac540 2024-08-17T02:46:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.972+0000 7fce4be00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4c0acd30 con 0x7fce4c069710 2024-08-17T02:46:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.972+0000 7fce4aa00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fce4c158bd0 0x7fce4c069000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57224/0 (socket says 172.21.15.16:57224) 2024-08-17T02:46:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.972+0000 7fce4aa00640 1 -- 172.21.15.16:0/2911604029 learned_addr learned my addr 172.21.15.16:0/2911604029 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.972+0000 7fce4a000640 1 --2- 172.21.15.16:0/2911604029 >> v2:172.21.15.142:3300/0 conn(0x7fce4c069710 0x7fce4c0ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.973+0000 7fce49600640 1 -- 172.21.15.16:0/2911604029 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120178280 0 0) 0x7fce4c0aec20 con 0x7fce4c158bd0 2024-08-17T02:46:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.973+0000 7fce49600640 1 -- 172.21.15.16:0/2911604029 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce300034e0 con 0x7fce4c158bd0 2024-08-17T02:46:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.973+0000 7fce4a000640 1 -- 172.21.15.16:0/2911604029 >> v1:172.21.15.142:6789/0 conn(0x7fce4c0ac540 legacy=0x7fce4c0af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.973+0000 7fce4a000640 1 -- 172.21.15.16:0/2911604029 >> v1:172.21.15.16:6789/0 conn(0x7fce4c158bd0 legacy=0x7fce4c069000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:16.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.973+0000 7fce4a000640 1 -- 172.21.15.16:0/2911604029 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0ae5d0 con 0x7fce4c069710 2024-08-17T02:46:16.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.974+0000 7fce4a000640 1 --2- 172.21.15.16:0/2911604029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce4c069710 0x7fce4c0ac000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fce38009850 tx=0x7fce380090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad8e1037dd601767 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:16.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.974+0000 7fce49600640 1 -- 172.21.15.16:0/2911604029 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3802ad00 con 0x7fce4c069710 2024-08-17T02:46:16.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.974+0000 7fce49600640 1 -- 172.21.15.16:0/2911604029 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3802ae60 con 0x7fce4c069710 2024-08-17T02:46:16.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.974+0000 7fce49600640 1 -- 172.21.15.16:0/2911604029 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38032650 con 0x7fce4c069710 2024-08-17T02:46:16.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.975+0000 7fce4be00640 1 -- 172.21.15.16:0/2911604029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce4c069710 msgr2=0x7fce4c0ac000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:16.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.975+0000 7fce4be00640 1 --2- 172.21.15.16:0/2911604029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce4c069710 0x7fce4c0ac000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fce38009850 tx=0x7fce380090f0 comp rx=0 tx=0).stop 2024-08-17T02:46:16.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.975+0000 7fce4be00640 1 -- 172.21.15.16:0/2911604029 shutdown_connections 2024-08-17T02:46:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.975+0000 7fce4be00640 1 --2- 172.21.15.16:0/2911604029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce4c069710 0x7fce4c0ac000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.975+0000 7fce4be00640 1 -- 172.21.15.16:0/2911604029 >> 172.21.15.16:0/2911604029 conn(0x7fce4c096a70 msgr2=0x7fce4c0cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.975+0000 7fce4be00640 1 -- 172.21.15.16:0/2911604029 shutdown_connections 2024-08-17T02:46:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.975+0000 7fce4be00640 1 -- 172.21.15.16:0/2911604029 wait complete. 2024-08-17T02:46:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.976+0000 7fce4be00640 1 Processor -- start 2024-08-17T02:46:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.976+0000 7fce4be00640 1 -- start start 2024-08-17T02:46:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.977+0000 7fce4be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce4c069710 0x7fce4c159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.977+0000 7fce4be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c0ac540 0x7fce4c09ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.977+0000 7fce4be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce4c158bd0 0x7fce4c1520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.977+0000 7fce4be00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c15be90 con 0x7fce4c0ac540 2024-08-17T02:46:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.977+0000 7fce4be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c1597c0 con 0x7fce4c069710 2024-08-17T02:46:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.977+0000 7fce4be00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c159aa0 con 0x7fce4c158bd0 2024-08-17T02:46:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c0ac540 0x7fce4c09ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:16.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4a000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c0ac540 0x7fce4c09ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43008/0 (socket says 172.21.15.16:43008) 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4a000640 1 -- 172.21.15.16:0/365256702 learned_addr learned my addr 172.21.15.16:0/365256702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4aa00640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce4c069710 0x7fce4c159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4b400640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce4c158bd0 0x7fce4c1520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4a000640 1 -- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce4c158bd0 msgr2=0x7fce4c1520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4a000640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce4c158bd0 0x7fce4c1520f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4a000640 1 -- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce4c069710 msgr2=0x7fce4c159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4a000640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce4c069710 0x7fce4c159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4a000640 1 -- 172.21.15.16:0/365256702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce38009390 con 0x7fce4c0ac540 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4aa00640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce4c069710 0x7fce4c159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4b400640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce4c158bd0 0x7fce4c1520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.978+0000 7fce4a000640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c0ac540 0x7fce4c09ae60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fce38009850 tx=0x7fce38032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.979+0000 7fce42000640 1 -- 172.21.15.16:0/365256702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce380316b0 con 0x7fce4c0ac540 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.979+0000 7fce42000640 1 -- 172.21.15.16:0/365256702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce38031830 con 0x7fce4c0ac540 2024-08-17T02:46:16.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.979+0000 7fce42000640 1 -- 172.21.15.16:0/365256702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38031bf0 con 0x7fce4c0ac540 2024-08-17T02:46:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.979+0000 7fce4be00640 1 -- 172.21.15.16:0/365256702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c159d80 con 0x7fce4c0ac540 2024-08-17T02:46:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.979+0000 7fce4be00640 1 -- 172.21.15.16:0/365256702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c15a340 con 0x7fce4c0ac540 2024-08-17T02:46:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.980+0000 7fce4be00640 1 -- 172.21.15.16:0/365256702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4c0973a0 con 0x7fce4c0ac540 2024-08-17T02:46:16.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.981+0000 7fce42000640 1 -- 172.21.15.16:0/365256702 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fce3803c400 con 0x7fce4c0ac540 2024-08-17T02:46:16.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.982+0000 7fce42000640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fce2c065150 0x7fce2c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:16.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.982+0000 7fce42000640 1 -- 172.21.15.16:0/365256702 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce380a24a0 con 0x7fce4c0ac540 2024-08-17T02:46:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.982+0000 7fce4aa00640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fce2c065150 0x7fce2c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:16.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.983+0000 7fce4aa00640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fce2c065150 0x7fce2c067600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce4c06c0a0 tx=0x7fce3c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:16.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:16.986+0000 7fce42000640 1 -- 172.21.15.16:0/365256702 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fce4c0973a0 con 0x7fce4c0ac540 2024-08-17T02:46:17.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.169+0000 7fce4be00640 1 -- 172.21.15.16:0/365256702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce4c0973a0 con 0x7fce4c0ac540 2024-08-17T02:46:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.170+0000 7fce42000640 1 -- 172.21.15.16:0/365256702 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15363 (secure 0 0 0) 0x7fce4c0973a0 con 0x7fce4c0ac540 2024-08-17T02:46:17.170 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:46:17.170 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:46:16.189871+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:46:14.322032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:46:15.170652+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:46:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.172+0000 7fce4be00640 1 -- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fce2c065150 msgr2=0x7fce2c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.172+0000 7fce4be00640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fce2c065150 0x7fce2c067600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce4c06c0a0 tx=0x7fce3c009040 comp rx=0 tx=0).stop 2024-08-17T02:46:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.172+0000 7fce4be00640 1 -- 172.21.15.16:0/365256702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c0ac540 msgr2=0x7fce4c09ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.172+0000 7fce4be00640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c0ac540 0x7fce4c09ae60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fce38009850 tx=0x7fce38032000 comp rx=0 tx=0).stop 2024-08-17T02:46:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.172+0000 7fce4be00640 1 -- 172.21.15.16:0/365256702 shutdown_connections 2024-08-17T02:46:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.172+0000 7fce4be00640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fce2c065150 0x7fce2c067600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.172+0000 7fce4be00640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fce4c158bd0 0x7fce4c1520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.172+0000 7fce4be00640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce4c0ac540 0x7fce4c09ae60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.172+0000 7fce4be00640 1 --2- 172.21.15.16:0/365256702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce4c069710 0x7fce4c159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.172+0000 7fce4be00640 1 -- 172.21.15.16:0/365256702 >> 172.21.15.16:0/365256702 conn(0x7fce4c096a70 msgr2=0x7fce4c0aa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.172+0000 7fce4be00640 1 -- 172.21.15.16:0/365256702 shutdown_connections 2024-08-17T02:46:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.172+0000 7fce4be00640 1 -- 172.21.15.16:0/365256702 wait complete. 2024-08-17T02:46:17.187 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:46:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.339+0000 7f1c0a600640 1 Processor -- start 2024-08-17T02:46:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.339+0000 7f1c0a600640 1 -- start start 2024-08-17T02:46:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.340+0000 7f1c0a600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1c0406df00 0x7f1c040adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.340+0000 7f1c0a600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c04155df0 0x7f1c040ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.340+0000 7f1c0a600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c04156670 con 0x7f1c04155a20 2024-08-17T02:46:17.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.340+0000 7f1c0a600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c040b0350 con 0x7f1c0406df00 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.340+0000 7f1c0a600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0406e5e0 con 0x7f1c04155df0 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.341+0000 7f1c03400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1c04155a20 0x7f1c0406d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57236/0 (socket says 172.21.15.16:57236) 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.341+0000 7f1c03400640 1 -- 172.21.15.16:0/1576706961 learned_addr learned my addr 172.21.15.16:0/1576706961 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.341+0000 7f1c02a00640 1 --2- 172.21.15.16:0/1576706961 >> v2:172.21.15.142:3300/0 conn(0x7f1c0406df00 0x7f1c040adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.341+0000 7f1c03e00640 1 --2- 172.21.15.16:0/1576706961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c04155df0 0x7f1c040ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.341+0000 7f1c02000640 1 -- 172.21.15.16:0/1576706961 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194228936 0 0) 0x7f1c04156670 con 0x7f1c04155a20 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.341+0000 7f1c02000640 1 -- 172.21.15.16:0/1576706961 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1be80034e0 con 0x7f1c04155a20 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.341+0000 7f1c02a00640 1 -- 172.21.15.16:0/1576706961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c04155df0 msgr2=0x7f1c040ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.341+0000 7f1c02a00640 1 --2- 172.21.15.16:0/1576706961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c04155df0 0x7f1c040ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.341+0000 7f1c02a00640 1 -- 172.21.15.16:0/1576706961 >> v1:172.21.15.16:6789/0 conn(0x7f1c04155a20 legacy=0x7f1c0406d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.341+0000 7f1c02a00640 1 -- 172.21.15.16:0/1576706961 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0406e8c0 con 0x7f1c0406df00 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.341+0000 7f1c03e00640 1 --2- 172.21.15.16:0/1576706961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c04155df0 0x7f1c040ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.343+0000 7f1c02a00640 1 --2- 172.21.15.16:0/1576706961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c0406df00 0x7f1c040adf10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009850 tx=0x7f1bf00090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e44a4ddc8356138b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:17.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.343+0000 7f1c02000640 1 -- 172.21.15.16:0/1576706961 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf002ad90 con 0x7f1c0406df00 2024-08-17T02:46:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.343+0000 7f1c02000640 1 -- 172.21.15.16:0/1576706961 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf0032440 con 0x7f1c0406df00 2024-08-17T02:46:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.343+0000 7f1c02000640 1 -- 172.21.15.16:0/1576706961 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0032800 con 0x7f1c0406df00 2024-08-17T02:46:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.343+0000 7f1c0a600640 1 -- 172.21.15.16:0/1576706961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c0406df00 msgr2=0x7f1c040adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.343+0000 7f1c0a600640 1 --2- 172.21.15.16:0/1576706961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c0406df00 0x7f1c040adf10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009850 tx=0x7f1bf00090f0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.343+0000 7f1c0a600640 1 -- 172.21.15.16:0/1576706961 shutdown_connections 2024-08-17T02:46:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.343+0000 7f1c0a600640 1 --2- 172.21.15.16:0/1576706961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c0406df00 0x7f1c040adf10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.343+0000 7f1c0a600640 1 --2- 172.21.15.16:0/1576706961 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c04155df0 0x7f1c040ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.344+0000 7f1c0a600640 1 -- 172.21.15.16:0/1576706961 >> 172.21.15.16:0/1576706961 conn(0x7f1c040cb9b0 msgr2=0x7f1c040cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.344+0000 7f1c0a600640 1 -- 172.21.15.16:0/1576706961 shutdown_connections 2024-08-17T02:46:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.344+0000 7f1c0a600640 1 -- 172.21.15.16:0/1576706961 wait complete. 2024-08-17T02:46:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.345+0000 7f1c0a600640 1 Processor -- start 2024-08-17T02:46:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.345+0000 7f1c0a600640 1 -- start start 2024-08-17T02:46:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.346+0000 7f1c0a600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c0406df00 0x7f1c04153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.346+0000 7f1c0a600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c04155a20 0x7f1c04136440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.346+0000 7f1c0a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c04155df0 0x7f1c04169a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.346+0000 7f1c0a600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c04153d80 con 0x7f1c04155df0 2024-08-17T02:46:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.346+0000 7f1c0a600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c04136980 con 0x7f1c0406df00 2024-08-17T02:46:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.346+0000 7f1c0a600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c04169f90 con 0x7f1c04155a20 2024-08-17T02:46:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.346+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c04155df0 0x7f1c04169a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.346+0000 7f1c03e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c04155df0 0x7f1c04169a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43016/0 (socket says 172.21.15.16:43016) 2024-08-17T02:46:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.346+0000 7f1c03e00640 1 -- 172.21.15.16:0/1588976315 learned_addr learned my addr 172.21.15.16:0/1588976315 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.347+0000 7f1c02a00640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c04155a20 0x7f1c04136440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.347+0000 7f1c03400640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c0406df00 0x7f1c04153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.347+0000 7f1c03e00640 1 -- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c04155a20 msgr2=0x7f1c04136440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.347+0000 7f1c03e00640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c04155a20 0x7f1c04136440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.347+0000 7f1c03e00640 1 -- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c0406df00 msgr2=0x7f1c04153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.347+0000 7f1c03e00640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c0406df00 0x7f1c04153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.347+0000 7f1c03e00640 1 -- 172.21.15.16:0/1588976315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc009a20 con 0x7f1c04155df0 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.347+0000 7f1c03e00640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c04155df0 0x7f1c04169a50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009f50 tx=0x7f1bfc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.347+0000 7f1bfaa00640 1 -- 172.21.15.16:0/1588976315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00cd90 con 0x7f1c04155df0 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.347+0000 7f1bfaa00640 1 -- 172.21.15.16:0/1588976315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc00f040 con 0x7f1c04155df0 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.347+0000 7f1bfaa00640 1 -- 172.21.15.16:0/1588976315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0146b0 con 0x7f1c04155df0 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.347+0000 7f1c0a600640 1 -- 172.21.15.16:0/1588976315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf0009390 con 0x7f1c04155df0 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.348+0000 7f1c0a600640 1 -- 172.21.15.16:0/1588976315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c041527a0 con 0x7f1c04155df0 2024-08-17T02:46:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.349+0000 7f1c0a600640 1 -- 172.21.15.16:0/1588976315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c04064db0 con 0x7f1c04155df0 2024-08-17T02:46:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.349+0000 7f1bfaa00640 1 -- 172.21.15.16:0/1588976315 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1bfc014a60 con 0x7f1c04155df0 2024-08-17T02:46:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.350+0000 7f1bfaa00640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1be4065150 0x7f1be4067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.350+0000 7f1bfaa00640 1 -- 172.21.15.16:0/1588976315 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1bfc083200 con 0x7f1c04155df0 2024-08-17T02:46:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.351+0000 7f1c03400640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1be4065150 0x7f1be4067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:17.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.351+0000 7f1c03400640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1be4065150 0x7f1be4067600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1c04154e30 tx=0x7f1bf4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:17.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.354+0000 7f1bfaa00640 1 -- 172.21.15.16:0/1588976315 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1bfc0512d0 con 0x7f1c04155df0 2024-08-17T02:46:17.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.536+0000 7f1c0a600640 1 -- 172.21.15.16:0/1588976315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c04154200 con 0x7f1c04155df0 2024-08-17T02:46:17.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.537+0000 7f1bfaa00640 1 -- 172.21.15.16:0/1588976315 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15363 (secure 0 0 0) 0x7f1bfc0548f0 con 0x7f1c04155df0 2024-08-17T02:46:17.537 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:46:17.538 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:46:16.189871+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:46:14.322032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:46:15.170652+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:46:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.539+0000 7f1c0a600640 1 -- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1be4065150 msgr2=0x7f1be4067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.539+0000 7f1c0a600640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1be4065150 0x7f1be4067600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1c04154e30 tx=0x7f1bf4009040 comp rx=0 tx=0).stop 2024-08-17T02:46:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.539+0000 7f1c0a600640 1 -- 172.21.15.16:0/1588976315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c04155df0 msgr2=0x7f1c04169a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.539+0000 7f1c0a600640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c04155df0 0x7f1c04169a50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009f50 tx=0x7f1bfc00b040 comp rx=0 tx=0).stop 2024-08-17T02:46:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.539+0000 7f1c0a600640 1 -- 172.21.15.16:0/1588976315 shutdown_connections 2024-08-17T02:46:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.539+0000 7f1c0a600640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1be4065150 0x7f1be4067600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.539+0000 7f1c0a600640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c04155df0 0x7f1c04169a50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.539+0000 7f1c0a600640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1c04155a20 0x7f1c04136440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.539+0000 7f1c0a600640 1 --2- 172.21.15.16:0/1588976315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c0406df00 0x7f1c04153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.539+0000 7f1c0a600640 1 -- 172.21.15.16:0/1588976315 >> 172.21.15.16:0/1588976315 conn(0x7f1c040cb9b0 msgr2=0x7f1c04154420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.539+0000 7f1c0a600640 1 -- 172.21.15.16:0/1588976315 shutdown_connections 2024-08-17T02:46:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.539+0000 7f1c0a600640 1 -- 172.21.15.16:0/1588976315 wait complete. 2024-08-17T02:46:17.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-08-17T02:46:17.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.709+0000 7f2529e00640 1 Processor -- start 2024-08-17T02:46:17.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.709+0000 7f2529e00640 1 -- start start 2024-08-17T02:46:17.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.710+0000 7f2529e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2524156c60 0x7f25240aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:17.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2529e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2524097180 0x7f25241607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:17.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2529e00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25240ad650 con 0x7f25240aade0 2024-08-17T02:46:17.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2529e00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25240ae490 con 0x7f2524156c60 2024-08-17T02:46:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2529e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25240ad880 con 0x7f2524097180 2024-08-17T02:46:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2522a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2524156c60 0x7f25240aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2522a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2524156c60 0x7f25240aa8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43032/0 (socket says 172.21.15.16:43032) 2024-08-17T02:46:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2522a00640 1 -- 172.21.15.16:0/394628452 learned_addr learned my addr 172.21.15.16:0/394628452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2522000640 1 --1- 172.21.15.16:0/394628452 >> v1:172.21.15.16:6789/0 conn(0x7f25240aade0 0x7f2524096a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57244/0 (socket says 172.21.15.16:57244) 2024-08-17T02:46:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2523400640 1 --2- 172.21.15.16:0/394628452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2524097180 0x7f25241607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2522a00640 1 -- 172.21.15.16:0/394628452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2524097180 msgr2=0x7f25241607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2522a00640 1 --2- 172.21.15.16:0/394628452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2524097180 0x7f25241607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2522a00640 1 -- 172.21.15.16:0/394628452 >> v1:172.21.15.16:6789/0 conn(0x7f25240aade0 legacy=0x7f2524096a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2522a00640 1 -- 172.21.15.16:0/394628452 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25240af930 con 0x7f2524156c60 2024-08-17T02:46:17.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.711+0000 7f2521600640 1 -- 172.21.15.16:0/394628452 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478002617 0 0) 0x7f25240ad650 con 0x7f25240aade0 2024-08-17T02:46:17.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.712+0000 7f2522a00640 1 --2- 172.21.15.16:0/394628452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2524156c60 0x7f25240aa8a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2514009910 tx=0x7f25140091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db6615eca8d18c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:17.713 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.713+0000 7f2521600640 1 -- 172.21.15.16:0/394628452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251402ae40 con 0x7f2524156c60 2024-08-17T02:46:17.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.713+0000 7f2521600640 1 -- 172.21.15.16:0/394628452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2514032440 con 0x7f2524156c60 2024-08-17T02:46:17.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.713+0000 7f2521600640 1 -- 172.21.15.16:0/394628452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25140327e0 con 0x7f2524156c60 2024-08-17T02:46:17.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.713+0000 7f2529e00640 1 -- 172.21.15.16:0/394628452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2524156c60 msgr2=0x7f25240aa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:17.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.713+0000 7f2529e00640 1 --2- 172.21.15.16:0/394628452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2524156c60 0x7f25240aa8a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2514009910 tx=0x7f25140091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.713+0000 7f2529e00640 1 -- 172.21.15.16:0/394628452 shutdown_connections 2024-08-17T02:46:17.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.713+0000 7f2529e00640 1 --2- 172.21.15.16:0/394628452 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2524097180 0x7f25241607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.713+0000 7f2529e00640 1 --2- 172.21.15.16:0/394628452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2524156c60 0x7f25240aa8a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.713+0000 7f2529e00640 1 -- 172.21.15.16:0/394628452 >> 172.21.15.16:0/394628452 conn(0x7f25240b2fd0 msgr2=0x7f25240b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:17.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.713+0000 7f2529e00640 1 -- 172.21.15.16:0/394628452 shutdown_connections 2024-08-17T02:46:17.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.713+0000 7f2529e00640 1 -- 172.21.15.16:0/394628452 wait complete. 2024-08-17T02:46:17.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.715+0000 7f2529e00640 1 Processor -- start 2024-08-17T02:46:17.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.715+0000 7f2529e00640 1 -- start start 2024-08-17T02:46:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2529e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2524097180 0x7f2524166ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2529e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25240aade0 0x7f25241496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2529e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2524156c60 0x7f2524157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2529e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25240b1ab0 con 0x7f2524097180 2024-08-17T02:46:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2529e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252414dcd0 con 0x7f2524156c60 2024-08-17T02:46:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2529e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252414d6c0 con 0x7f25240aade0 2024-08-17T02:46:17.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2522a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2524097180 0x7f2524166ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2522a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2524097180 0x7f2524166ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43036/0 (socket says 172.21.15.16:43036) 2024-08-17T02:46:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2522a00640 1 -- 172.21.15.16:0/90865824 learned_addr learned my addr 172.21.15.16:0/90865824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2522000640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25240aade0 0x7f25241496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2523400640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2524156c60 0x7f2524157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2522a00640 1 -- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25240aade0 msgr2=0x7f25241496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2522a00640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25240aade0 0x7f25241496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2522a00640 1 -- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2524156c60 msgr2=0x7f2524157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2522a00640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2524156c60 0x7f2524157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.716+0000 7f2522a00640 1 -- 172.21.15.16:0/90865824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2514009450 con 0x7f2524097180 2024-08-17T02:46:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.717+0000 7f2522a00640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2524097180 0x7f2524166ee0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2514009910 tx=0x7f2514032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.717+0000 7f251a000640 1 -- 172.21.15.16:0/90865824 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514000d10 con 0x7f2524097180 2024-08-17T02:46:17.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.717+0000 7f251a000640 1 -- 172.21.15.16:0/90865824 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2514031e00 con 0x7f2524097180 2024-08-17T02:46:17.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.717+0000 7f251a000640 1 -- 172.21.15.16:0/90865824 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251403d690 con 0x7f2524097180 2024-08-17T02:46:17.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.717+0000 7f2529e00640 1 -- 172.21.15.16:0/90865824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25240b17d0 con 0x7f2524097180 2024-08-17T02:46:17.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.717+0000 7f2529e00640 1 -- 172.21.15.16:0/90865824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2524167420 con 0x7f2524097180 2024-08-17T02:46:17.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.718+0000 7f2529e00640 1 -- 172.21.15.16:0/90865824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2524071050 con 0x7f2524097180 2024-08-17T02:46:17.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.719+0000 7f251a000640 1 -- 172.21.15.16:0/90865824 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f2514039040 con 0x7f2524097180 2024-08-17T02:46:17.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.720+0000 7f251a000640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2504065150 0x7f2504067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:17.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.720+0000 7f251a000640 1 -- 172.21.15.16:0/90865824 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f251402ae40 con 0x7f2524097180 2024-08-17T02:46:17.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.720+0000 7f2522000640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2504065150 0x7f2504067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:17.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.721+0000 7f2522000640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2504065150 0x7f2504067600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f252416a3e0 tx=0x7f2510007b60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:17.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.724+0000 7f251a000640 1 -- 172.21.15.16:0/90865824 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2514070510 con 0x7f2524097180 2024-08-17T02:46:17.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:17.946+0000 7f2529e00640 1 -- 172.21.15.16:0/90865824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f252409b950 con 0x7f2524097180 2024-08-17T02:46:18.722 INFO:tasks.ceph.mds.a.smithi016.stderr:2024-08-17T02:46:18.721+0000 7f06c6400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-17T02:46:18.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.782+0000 7f251a000640 1 -- 172.21.15.16:0/90865824 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2514073b30 con 0x7f2524097180 2024-08-17T02:46:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.785+0000 7f2529e00640 1 -- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2504065150 msgr2=0x7f2504067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.785+0000 7f2529e00640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2504065150 0x7f2504067600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f252416a3e0 tx=0x7f2510007b60 comp rx=0 tx=0).stop 2024-08-17T02:46:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.785+0000 7f2529e00640 1 -- 172.21.15.16:0/90865824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2524097180 msgr2=0x7f2524166ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.785+0000 7f2529e00640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2524097180 0x7f2524166ee0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2514009910 tx=0x7f2514032000 comp rx=0 tx=0).stop 2024-08-17T02:46:18.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.785+0000 7f2529e00640 1 -- 172.21.15.16:0/90865824 shutdown_connections 2024-08-17T02:46:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.785+0000 7f2529e00640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2504065150 0x7f2504067600 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.785+0000 7f2529e00640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2524156c60 0x7f2524157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.785+0000 7f2529e00640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f25240aade0 0x7f25241496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.785+0000 7f2529e00640 1 --2- 172.21.15.16:0/90865824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2524097180 0x7f2524166ee0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.785+0000 7f2529e00640 1 -- 172.21.15.16:0/90865824 >> 172.21.15.16:0/90865824 conn(0x7f25240b2fd0 msgr2=0x7f25240ab1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.785+0000 7f2529e00640 1 -- 172.21.15.16:0/90865824 shutdown_connections 2024-08-17T02:46:18.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.785+0000 7f2529e00640 1 -- 172.21.15.16:0/90865824 wait complete. 2024-08-17T02:46:18.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-08-17T02:46:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.963+0000 7fcc65600640 1 Processor -- start 2024-08-17T02:46:18.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.963+0000 7fcc65600640 1 -- start start 2024-08-17T02:46:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.964+0000 7fcc65600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc60155a80 0x7fcc6006dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.964+0000 7fcc65600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcc600c7db0 0x7fcc600c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:18.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.965+0000 7fcc65600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc601598f0 con 0x7fcc60155e50 2024-08-17T02:46:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.965+0000 7fcc65600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc60159bd0 con 0x7fcc60155a80 2024-08-17T02:46:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.965+0000 7fcc65600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc60159eb0 con 0x7fcc600c7db0 2024-08-17T02:46:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.965+0000 7fcc5ea00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc60155a80 0x7fcc6006dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.965+0000 7fcc5ea00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc60155a80 0x7fcc6006dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43046/0 (socket says 172.21.15.16:43046) 2024-08-17T02:46:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.965+0000 7fcc5ea00640 1 -- 172.21.15.16:0/3529544586 learned_addr learned my addr 172.21.15.16:0/3529544586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.965+0000 7fcc5f400640 1 --1- 172.21.15.16:0/3529544586 >> v1:172.21.15.16:6789/0 conn(0x7fcc60155e50 0x7fcc6006c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57252/0 (socket says 172.21.15.16:57252) 2024-08-17T02:46:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.965+0000 7fcc5e000640 1 --2- 172.21.15.16:0/3529544586 >> v2:172.21.15.142:3300/0 conn(0x7fcc600c7db0 0x7fcc600c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.965+0000 7fcc5ea00640 1 -- 172.21.15.16:0/3529544586 >> v2:172.21.15.142:3300/0 conn(0x7fcc600c7db0 msgr2=0x7fcc600c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.965+0000 7fcc5ea00640 1 --2- 172.21.15.16:0/3529544586 >> v2:172.21.15.142:3300/0 conn(0x7fcc600c7db0 0x7fcc600c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.965+0000 7fcc5ea00640 1 -- 172.21.15.16:0/3529544586 >> v1:172.21.15.16:6789/0 conn(0x7fcc60155e50 legacy=0x7fcc6006c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:18.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.965+0000 7fcc5ea00640 1 -- 172.21.15.16:0/3529544586 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6015a190 con 0x7fcc60155a80 2024-08-17T02:46:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.966+0000 7fcc5ea00640 1 --2- 172.21.15.16:0/3529544586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60155a80 0x7fcc6006dc00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcc6006c750 tx=0x7fcc500091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40814967cdf0772e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.967+0000 7fcc5d600640 1 -- 172.21.15.16:0/3529544586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50032440 con 0x7fcc60155a80 2024-08-17T02:46:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.967+0000 7fcc5d600640 1 -- 172.21.15.16:0/3529544586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc500325a0 con 0x7fcc60155a80 2024-08-17T02:46:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.967+0000 7fcc5d600640 1 -- 172.21.15.16:0/3529544586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc50032980 con 0x7fcc60155a80 2024-08-17T02:46:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.967+0000 7fcc65600640 1 -- 172.21.15.16:0/3529544586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60155a80 msgr2=0x7fcc6006dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.967+0000 7fcc65600640 1 --2- 172.21.15.16:0/3529544586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60155a80 0x7fcc6006dc00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcc6006c750 tx=0x7fcc500091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.967+0000 7fcc65600640 1 -- 172.21.15.16:0/3529544586 shutdown_connections 2024-08-17T02:46:18.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.967+0000 7fcc65600640 1 --2- 172.21.15.16:0/3529544586 >> v2:172.21.15.142:3300/0 conn(0x7fcc600c7db0 0x7fcc600c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.967+0000 7fcc65600640 1 --2- 172.21.15.16:0/3529544586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60155a80 0x7fcc6006dc00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.967+0000 7fcc65600640 1 -- 172.21.15.16:0/3529544586 >> 172.21.15.16:0/3529544586 conn(0x7fcc60067750 msgr2=0x7fcc600c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.967+0000 7fcc65600640 1 -- 172.21.15.16:0/3529544586 shutdown_connections 2024-08-17T02:46:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.967+0000 7fcc65600640 1 -- 172.21.15.16:0/3529544586 wait complete. 2024-08-17T02:46:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.969+0000 7fcc65600640 1 Processor -- start 2024-08-17T02:46:18.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.969+0000 7fcc65600640 1 -- start start 2024-08-17T02:46:18.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc65600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc600c7db0 0x7fcc6016a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc65600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc60155a80 0x7fcc60142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc65600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60155e50 0x7fcc601587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc65600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6016a550 con 0x7fcc60155e50 2024-08-17T02:46:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc65600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc60157090 con 0x7fcc600c7db0 2024-08-17T02:46:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc65600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc60157370 con 0x7fcc60155a80 2024-08-17T02:46:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc5f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60155e50 0x7fcc601587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc5f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60155e50 0x7fcc601587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43060/0 (socket says 172.21.15.16:43060) 2024-08-17T02:46:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc5f400640 1 -- 172.21.15.16:0/180053359 learned_addr learned my addr 172.21.15.16:0/180053359 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc5ea00640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc600c7db0 0x7fcc6016a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc5e000640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc60155a80 0x7fcc60142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:18.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc5f400640 1 -- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc60155a80 msgr2=0x7fcc60142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.970+0000 7fcc5f400640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc60155a80 0x7fcc60142280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.971+0000 7fcc5f400640 1 -- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc600c7db0 msgr2=0x7fcc6016a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.971+0000 7fcc5f400640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc600c7db0 0x7fcc6016a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.971+0000 7fcc5f400640 1 -- 172.21.15.16:0/180053359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50009450 con 0x7fcc60155e50 2024-08-17T02:46:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.971+0000 7fcc5ea00640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc600c7db0 0x7fcc6016a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:46:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.971+0000 7fcc5e000640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc60155a80 0x7fcc60142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:46:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.971+0000 7fcc5f400640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60155e50 0x7fcc601587a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800c840 tx=0x7fcc580015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.971+0000 7fcc56000640 1 -- 172.21.15.16:0/180053359 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800ce30 con 0x7fcc60155e50 2024-08-17T02:46:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.971+0000 7fcc56000640 1 -- 172.21.15.16:0/180053359 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc58005d60 con 0x7fcc60155e50 2024-08-17T02:46:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.971+0000 7fcc65600640 1 -- 172.21.15.16:0/180053359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc60157650 con 0x7fcc60155e50 2024-08-17T02:46:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.971+0000 7fcc56000640 1 -- 172.21.15.16:0/180053359 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800a810 con 0x7fcc60155e50 2024-08-17T02:46:18.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.971+0000 7fcc65600640 1 -- 172.21.15.16:0/180053359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc601427c0 con 0x7fcc60155e50 2024-08-17T02:46:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.972+0000 7fcc65600640 1 -- 172.21.15.16:0/180053359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc600c9430 con 0x7fcc60155e50 2024-08-17T02:46:18.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.973+0000 7fcc56000640 1 -- 172.21.15.16:0/180053359 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fcc5800ac40 con 0x7fcc60155e50 2024-08-17T02:46:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.974+0000 7fcc56000640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcc40065150 0x7fcc40067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.974+0000 7fcc56000640 1 -- 172.21.15.16:0/180053359 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcc58082190 con 0x7fcc60155e50 2024-08-17T02:46:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.974+0000 7fcc5ea00640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcc40065150 0x7fcc40067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:18.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.975+0000 7fcc5ea00640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcc40065150 0x7fcc40067600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcc50034000 tx=0x7fcc50032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:18.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:18.978+0000 7fcc56000640 1 -- 172.21.15.16:0/180053359 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcc600c9430 con 0x7fcc60155e50 2024-08-17T02:46:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.198+0000 7fcc65600640 1 -- 172.21.15.16:0/180053359 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcc60142df0 con 0x7fcc60155e50 2024-08-17T02:46:19.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.783+0000 7fcc56000640 1 -- 172.21.15.16:0/180053359 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcc60142df0 con 0x7fcc60155e50 2024-08-17T02:46:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.786+0000 7fcc65600640 1 -- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcc40065150 msgr2=0x7fcc40067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.786+0000 7fcc65600640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcc40065150 0x7fcc40067600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcc50034000 tx=0x7fcc50032000 comp rx=0 tx=0).stop 2024-08-17T02:46:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.786+0000 7fcc65600640 1 -- 172.21.15.16:0/180053359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60155e50 msgr2=0x7fcc601587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.786+0000 7fcc65600640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60155e50 0x7fcc601587a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800c840 tx=0x7fcc580015f0 comp rx=0 tx=0).stop 2024-08-17T02:46:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.786+0000 7fcc65600640 1 -- 172.21.15.16:0/180053359 shutdown_connections 2024-08-17T02:46:19.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.786+0000 7fcc65600640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcc40065150 0x7fcc40067600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.786+0000 7fcc65600640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc60155e50 0x7fcc601587a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.786+0000 7fcc65600640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc60155a80 0x7fcc60142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.786+0000 7fcc65600640 1 --2- 172.21.15.16:0/180053359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc600c7db0 0x7fcc6016a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.786+0000 7fcc65600640 1 -- 172.21.15.16:0/180053359 >> 172.21.15.16:0/180053359 conn(0x7fcc60067750 msgr2=0x7fcc60068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.786+0000 7fcc65600640 1 -- 172.21.15.16:0/180053359 shutdown_connections 2024-08-17T02:46:19.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.786+0000 7fcc65600640 1 -- 172.21.15.16:0/180053359 wait complete. 2024-08-17T02:46:19.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:46:19.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:46:19.854 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:46:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.956+0000 7f7741200640 1 Processor -- start 2024-08-17T02:46:19.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.956+0000 7f7741200640 1 -- start start 2024-08-17T02:46:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.957+0000 7f7741200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f773c140580 0x7f773c0c9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.957+0000 7f7741200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773c0caa80 con 0x7f773c0b1440 2024-08-17T02:46:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.957+0000 7f7741200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773c0c9e70 con 0x7f773c1401b0 2024-08-17T02:46:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.957+0000 7f7741200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773c0c9c90 con 0x7f773c140580 2024-08-17T02:46:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.958+0000 7f7739400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f773c0b1440 0x7f773c0c9040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57264/0 (socket says 172.21.15.16:57264) 2024-08-17T02:46:19.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.958+0000 7f7739400640 1 -- 172.21.15.16:0/1639396317 learned_addr learned my addr 172.21.15.16:0/1639396317 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.958+0000 7f773a800640 1 --2- 172.21.15.16:0/1639396317 >> v2:172.21.15.142:3300/0 conn(0x7f773c140580 0x7f773c0c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.958+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025534099 0 0) 0x7f773c0caa80 con 0x7f773c0b1440 2024-08-17T02:46:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.958+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77200034e0 con 0x7f773c0b1440 2024-08-17T02:46:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.958+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3156711334 0 0) 0x7f77200034e0 con 0x7f773c0b1440 2024-08-17T02:46:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.958+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 >> v2:172.21.15.142:3300/0 conn(0x7f773c140580 msgr2=0x7f773c0c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.958+0000 7f7738a00640 1 --2- 172.21.15.16:0/1639396317 >> v2:172.21.15.142:3300/0 conn(0x7f773c140580 0x7f773c0c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.958+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 >> v1:172.21.15.142:6789/0 conn(0x7f773c1401b0 legacy=0x7f773c0b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.958+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c0ca150 con 0x7f773c0b1440 2024-08-17T02:46:19.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.958+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7f7728001470 con 0x7f773c0b1440 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 >> v1:172.21.15.16:6789/0 conn(0x7f773c0b1440 legacy=0x7f773c0c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7738a00640 1 --2- 172.21.15.16:0/1639396317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77200039d0 0x7f7720003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7738a00640 1 --2- 172.21.15.16:0/1639396317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7720004780 0x7f7720004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7738a00640 1 --2- 172.21.15.16:0/1639396317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720005130 0x7f772000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0c9c90 con 0x7f7720005130 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772000bfc0 con 0x7f77200039d0 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772000c180 con 0x7f7720004780 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7739e00640 1 --2- 172.21.15.16:0/1639396317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77200039d0 0x7f7720003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f773a800640 1 --2- 172.21.15.16:0/1639396317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720005130 0x7f772000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7739400640 1 --2- 172.21.15.16:0/1639396317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7720004780 0x7f7720004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7739e00640 1 -- 172.21.15.16:0/1639396317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7720004780 msgr2=0x7f7720004bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7739e00640 1 --2- 172.21.15.16:0/1639396317 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7720004780 0x7f7720004bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7739e00640 1 -- 172.21.15.16:0/1639396317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720005130 msgr2=0x7f772000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:19.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7739e00640 1 --2- 172.21.15.16:0/1639396317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7720005130 0x7f772000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f7739e00640 1 -- 172.21.15.16:0/1639396317 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77200034e0 con 0x7f77200039d0 2024-08-17T02:46:19.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.959+0000 7f773a800640 1 -- 172.21.15.16:0/1639396317 reap_dead start 2024-08-17T02:46:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.960+0000 7f7739e00640 1 --2- 172.21.15.16:0/1639396317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77200039d0 0x7f7720003da0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f772c0019e0 tx=0x7f772c007ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd0bacb81eed28 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.961+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c02aa60 con 0x7f77200039d0 2024-08-17T02:46:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.961+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c02abe0 con 0x7f77200039d0 2024-08-17T02:46:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.961+0000 7f7738a00640 1 -- 172.21.15.16:0/1639396317 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c0323f0 con 0x7f77200039d0 2024-08-17T02:46:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.961+0000 7f7741200640 1 -- 172.21.15.16:0/1639396317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77200039d0 msgr2=0x7f7720003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.961+0000 7f7741200640 1 --2- 172.21.15.16:0/1639396317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77200039d0 0x7f7720003da0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f772c0019e0 tx=0x7f772c007ca0 comp rx=0 tx=0).stop 2024-08-17T02:46:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.961+0000 7f7741200640 1 -- 172.21.15.16:0/1639396317 shutdown_connections 2024-08-17T02:46:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.961+0000 7f7741200640 1 --2- 172.21.15.16:0/1639396317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77200039d0 0x7f7720003da0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.961+0000 7f7741200640 1 -- 172.21.15.16:0/1639396317 >> 172.21.15.16:0/1639396317 conn(0x7f773c0cb9b0 msgr2=0x7f773c0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.962+0000 7f7741200640 1 -- 172.21.15.16:0/1639396317 shutdown_connections 2024-08-17T02:46:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.962+0000 7f7741200640 1 -- 172.21.15.16:0/1639396317 wait complete. 2024-08-17T02:46:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.963+0000 7f7741200640 1 Processor -- start 2024-08-17T02:46:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.963+0000 7f7741200640 1 -- start start 2024-08-17T02:46:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.964+0000 7f7741200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77200039d0 0x7f773c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.964+0000 7f7741200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f773c0c1700 0x7f773c0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.964+0000 7f7741200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f773c169590 0x7f773c0aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.964+0000 7f7741200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c169100 con 0x7f77200039d0 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.964+0000 7f7741200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0c1c70 con 0x7f773c169590 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.964+0000 7f7741200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c0c1f50 con 0x7f773c0c1700 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.964+0000 7f7739e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77200039d0 0x7f773c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.964+0000 7f7739e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77200039d0 0x7f773c15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43078/0 (socket says 172.21.15.16:43078) 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.964+0000 7f7739e00640 1 -- 172.21.15.16:0/1719973737 learned_addr learned my addr 172.21.15.16:0/1719973737 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f7739400640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f773c0c1700 0x7f773c0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f773a800640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f773c169590 0x7f773c0aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f7739e00640 1 -- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f773c0c1700 msgr2=0x7f773c0c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f7739e00640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f773c0c1700 0x7f773c0c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f7739e00640 1 -- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f773c169590 msgr2=0x7f773c0aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f7739e00640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f773c169590 0x7f773c0aba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f7739e00640 1 -- 172.21.15.16:0/1719973737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c168d90 con 0x7f77200039d0 2024-08-17T02:46:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f773a800640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f773c169590 0x7f773c0aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f7739e00640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77200039d0 0x7f773c15aa80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f772c02a660 tx=0x7f772c003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f7731600640 1 -- 172.21.15.16:0/1719973737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c0073f0 con 0x7f77200039d0 2024-08-17T02:46:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f7731600640 1 -- 172.21.15.16:0/1719973737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c007550 con 0x7f77200039d0 2024-08-17T02:46:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f7731600640 1 -- 172.21.15.16:0/1719973737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c0076c0 con 0x7f77200039d0 2024-08-17T02:46:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.965+0000 7f7741200640 1 -- 172.21.15.16:0/1719973737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773c0c7ae0 con 0x7f77200039d0 2024-08-17T02:46:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.966+0000 7f7741200640 1 -- 172.21.15.16:0/1719973737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773c169a00 con 0x7f77200039d0 2024-08-17T02:46:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.967+0000 7f7741200640 1 -- 172.21.15.16:0/1719973737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773c168f20 con 0x7f77200039d0 2024-08-17T02:46:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.967+0000 7f7731600640 1 -- 172.21.15.16:0/1719973737 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f772c006c70 con 0x7f77200039d0 2024-08-17T02:46:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.968+0000 7f7731600640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f771c065180 0x7f771c067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.968+0000 7f7731600640 1 -- 172.21.15.16:0/1719973737 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f772c0a1a00 con 0x7f77200039d0 2024-08-17T02:46:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.969+0000 7f7739400640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f771c065180 0x7f771c067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.969+0000 7f7739400640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f771c065180 0x7f771c067630 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f773c13f640 tx=0x7f7728001df0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:19.972+0000 7f7731600640 1 -- 172.21.15.16:0/1719973737 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f772c06fc10 con 0x7f77200039d0 2024-08-17T02:46:20.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.191+0000 7f7741200640 1 -- 172.21.15.16:0/1719973737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f773c169b90 con 0x7f77200039d0 2024-08-17T02:46:20.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.192+0000 7f7731600640 1 -- 172.21.15.16:0/1719973737 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f772c041090 con 0x7f77200039d0 2024-08-17T02:46:20.192 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:46:20.192 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2889208700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2889208700},{"type":"v1","addr":"172.21.15.142:6837","nonce":2889208700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4287834697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4287834697},{"type":"v1","addr":"172.21.15.142:6835","nonce":4287834697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3115083427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3115083427},{"type":"v1","addr":"172.21.15.16:6834","nonce":3115083427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:16.190199+0000","modified":"2024-08-17T02:46:19.780153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6835/219684854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":219684854},{"type":"v1","addr":"172.21.15.16:6835","nonce":219684854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-08-17T02:46:20.193 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 18 2024-08-17T02:46:20.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.194+0000 7f7741200640 1 -- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f771c065180 msgr2=0x7f771c067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.194+0000 7f7741200640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f771c065180 0x7f771c067630 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f773c13f640 tx=0x7f7728001df0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.194+0000 7f7741200640 1 -- 172.21.15.16:0/1719973737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77200039d0 msgr2=0x7f773c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.194+0000 7f7741200640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77200039d0 0x7f773c15aa80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f772c02a660 tx=0x7f772c003000 comp rx=0 tx=0).stop 2024-08-17T02:46:20.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.194+0000 7f7741200640 1 -- 172.21.15.16:0/1719973737 shutdown_connections 2024-08-17T02:46:20.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.194+0000 7f7741200640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f771c065180 0x7f771c067630 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.194+0000 7f7741200640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f773c169590 0x7f773c0aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.194+0000 7f7741200640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f773c0c1700 0x7f773c0c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.194+0000 7f7741200640 1 --2- 172.21.15.16:0/1719973737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77200039d0 0x7f773c15aa80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.194+0000 7f7741200640 1 -- 172.21.15.16:0/1719973737 >> 172.21.15.16:0/1719973737 conn(0x7f773c0cb9b0 msgr2=0x7f773c0aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:20.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.194+0000 7f7741200640 1 -- 172.21.15.16:0/1719973737 shutdown_connections 2024-08-17T02:46:20.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.194+0000 7f7741200640 1 -- 172.21.15.16:0/1719973737 wait complete. 2024-08-17T02:46:20.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:46:20.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.402+0000 7f21efe00640 1 Processor -- start 2024-08-17T02:46:20.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.403+0000 7f21efe00640 1 -- start start 2024-08-17T02:46:20.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.404+0000 7f21efe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21f00aed50 0x7f21f00ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:20.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.404+0000 7f21efe00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f21f00c5c40 0x7f21f00caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:20.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.404+0000 7f21efe00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f00cc890 con 0x7f21f00ce6d0 2024-08-17T02:46:20.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.404+0000 7f21efe00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f00ceb20 con 0x7f21f00aed50 2024-08-17T02:46:20.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.404+0000 7f21efe00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f00cb4c0 con 0x7f21f00c5c40 2024-08-17T02:46:20.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.404+0000 7f21eea00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21f00aed50 0x7f21f00ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:20.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.404+0000 7f21eea00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21f00aed50 0x7f21f00ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43092/0 (socket says 172.21.15.16:43092) 2024-08-17T02:46:20.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.404+0000 7f21eea00640 1 -- 172.21.15.16:0/1994093967 learned_addr learned my addr 172.21.15.16:0/1994093967 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:20.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.404+0000 7f21e7e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f21f00ce6d0 0x7f21f00ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57280/0 (socket says 172.21.15.16:57280) 2024-08-17T02:46:20.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.404+0000 7f21ef400640 1 --2- 172.21.15.16:0/1994093967 >> v2:172.21.15.142:3300/0 conn(0x7f21f00c5c40 0x7f21f00caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:20.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.405+0000 7f21eea00640 1 -- 172.21.15.16:0/1994093967 >> v2:172.21.15.142:3300/0 conn(0x7f21f00c5c40 msgr2=0x7f21f00caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:20.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.405+0000 7f21eea00640 1 --2- 172.21.15.16:0/1994093967 >> v2:172.21.15.142:3300/0 conn(0x7f21f00c5c40 0x7f21f00caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.405+0000 7f21eea00640 1 -- 172.21.15.16:0/1994093967 >> v1:172.21.15.16:6789/0 conn(0x7f21f00ce6d0 legacy=0x7f21f00ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.405+0000 7f21eea00640 1 -- 172.21.15.16:0/1994093967 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f00cb7a0 con 0x7f21f00aed50 2024-08-17T02:46:20.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.405+0000 7f21ee000640 1 -- 172.21.15.16:0/1994093967 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283836551 0 0) 0x7f21f00cc890 con 0x7f21f00ce6d0 2024-08-17T02:46:20.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.406+0000 7f21eea00640 1 --2- 172.21.15.16:0/1994093967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f00aed50 0x7f21f00ce190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f21f0089340 tx=0x7f21e00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e89035c305a7af server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:20.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.406+0000 7f21ee000640 1 -- 172.21.15.16:0/1994093967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e002ae80 con 0x7f21f00aed50 2024-08-17T02:46:20.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.406+0000 7f21ee000640 1 -- 172.21.15.16:0/1994093967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e0032440 con 0x7f21f00aed50 2024-08-17T02:46:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.406+0000 7f21ee000640 1 -- 172.21.15.16:0/1994093967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e00327c0 con 0x7f21f00aed50 2024-08-17T02:46:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.406+0000 7f21efe00640 1 -- 172.21.15.16:0/1994093967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f00aed50 msgr2=0x7f21f00ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.406+0000 7f21efe00640 1 --2- 172.21.15.16:0/1994093967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f00aed50 0x7f21f00ce190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f21f0089340 tx=0x7f21e00091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.407+0000 7f21efe00640 1 -- 172.21.15.16:0/1994093967 shutdown_connections 2024-08-17T02:46:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.407+0000 7f21efe00640 1 --2- 172.21.15.16:0/1994093967 >> v2:172.21.15.142:3300/0 conn(0x7f21f00c5c40 0x7f21f00caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.407+0000 7f21efe00640 1 --2- 172.21.15.16:0/1994093967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f00aed50 0x7f21f00ce190 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.407+0000 7f21efe00640 1 -- 172.21.15.16:0/1994093967 >> 172.21.15.16:0/1994093967 conn(0x7f21f0098e30 msgr2=0x7f21f006eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.407+0000 7f21efe00640 1 -- 172.21.15.16:0/1994093967 shutdown_connections 2024-08-17T02:46:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.407+0000 7f21efe00640 1 -- 172.21.15.16:0/1994093967 wait complete. 2024-08-17T02:46:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.408+0000 7f21efe00640 1 Processor -- start 2024-08-17T02:46:20.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.408+0000 7f21efe00640 1 -- start start 2024-08-17T02:46:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.409+0000 7f21efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f00aed50 0x7f21f00b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.409+0000 7f21efe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21f00c5c40 0x7f21f0168e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.409+0000 7f21efe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21f00ce6d0 0x7f21f01693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.409+0000 7f21efe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f00b4130 con 0x7f21f00aed50 2024-08-17T02:46:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.409+0000 7f21efe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f00b3520 con 0x7f21f00ce6d0 2024-08-17T02:46:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.409+0000 7f21efe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f00b3800 con 0x7f21f00c5c40 2024-08-17T02:46:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.409+0000 7f21eea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f00aed50 0x7f21f00b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.409+0000 7f21eea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f00aed50 0x7f21f00b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43106/0 (socket says 172.21.15.16:43106) 2024-08-17T02:46:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.409+0000 7f21eea00640 1 -- 172.21.15.16:0/2433033914 learned_addr learned my addr 172.21.15.16:0/2433033914 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.409+0000 7f21ef400640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21f00ce6d0 0x7f21f01693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:20.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.409+0000 7f21e7e00640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21f00c5c40 0x7f21f0168e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.410+0000 7f21eea00640 1 -- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21f00c5c40 msgr2=0x7f21f0168e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.410+0000 7f21eea00640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21f00c5c40 0x7f21f0168e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.410+0000 7f21eea00640 1 -- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21f00ce6d0 msgr2=0x7f21f01693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.410+0000 7f21eea00640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21f00ce6d0 0x7f21f01693c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.410+0000 7f21eea00640 1 -- 172.21.15.16:0/2433033914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e0009450 con 0x7f21f00aed50 2024-08-17T02:46:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.410+0000 7f21ef400640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21f00ce6d0 0x7f21f01693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:46:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.410+0000 7f21eea00640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f00aed50 0x7f21f00b51e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009210 tx=0x7f21e0032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.410+0000 7f21e6000640 1 -- 172.21.15.16:0/2433033914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e0031de0 con 0x7f21f00aed50 2024-08-17T02:46:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.410+0000 7f21e6000640 1 -- 172.21.15.16:0/2433033914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e0000c40 con 0x7f21f00aed50 2024-08-17T02:46:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.410+0000 7f21e6000640 1 -- 172.21.15.16:0/2433033914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e003d690 con 0x7f21f00aed50 2024-08-17T02:46:20.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.410+0000 7f21efe00640 1 -- 172.21.15.16:0/2433033914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f00b3ae0 con 0x7f21f00aed50 2024-08-17T02:46:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.410+0000 7f21efe00640 1 -- 172.21.15.16:0/2433033914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f00b5720 con 0x7f21f00aed50 2024-08-17T02:46:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.412+0000 7f21efe00640 1 -- 172.21.15.16:0/2433033914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f0071050 con 0x7f21f00aed50 2024-08-17T02:46:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.412+0000 7f21e6000640 1 -- 172.21.15.16:0/2433033914 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f21e0039040 con 0x7f21f00aed50 2024-08-17T02:46:20.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.413+0000 7f21e6000640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f21cc065180 0x7f21cc067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:20.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.413+0000 7f21e6000640 1 -- 172.21.15.16:0/2433033914 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f21e00a22a0 con 0x7f21f00aed50 2024-08-17T02:46:20.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.414+0000 7f21e7e00640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f21cc065180 0x7f21cc067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:20.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.414+0000 7f21e7e00640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f21cc065180 0x7f21cc067630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21f0068080 tx=0x7f21d8007cc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:20.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.417+0000 7f21e6000640 1 -- 172.21.15.16:0/2433033914 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f21e0070430 con 0x7f21f00aed50 2024-08-17T02:46:20.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.601+0000 7f21efe00640 1 -- 172.21.15.16:0/2433033914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21f00d00a0 con 0x7f21f00aed50 2024-08-17T02:46:20.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.602+0000 7f21e6000640 1 -- 172.21.15.16:0/2433033914 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+15350 (secure 0 0 0) 0x7f21e0073a50 con 0x7f21f00aed50 2024-08-17T02:46:20.602 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:46:20.602 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":33,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:46:17.768576+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:46:14.322032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:46:15.170652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:46:20.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.604+0000 7f21efe00640 1 -- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f21cc065180 msgr2=0x7f21cc067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.604+0000 7f21efe00640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f21cc065180 0x7f21cc067630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21f0068080 tx=0x7f21d8007cc0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.604+0000 7f21efe00640 1 -- 172.21.15.16:0/2433033914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f00aed50 msgr2=0x7f21f00b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.604+0000 7f21efe00640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f00aed50 0x7f21f00b51e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009210 tx=0x7f21e0032000 comp rx=0 tx=0).stop 2024-08-17T02:46:20.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.605+0000 7f21efe00640 1 -- 172.21.15.16:0/2433033914 shutdown_connections 2024-08-17T02:46:20.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.605+0000 7f21efe00640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f21cc065180 0x7f21cc067630 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.605+0000 7f21efe00640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21f00ce6d0 0x7f21f01693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.605+0000 7f21efe00640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21f00c5c40 0x7f21f0168e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.605+0000 7f21efe00640 1 --2- 172.21.15.16:0/2433033914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f00aed50 0x7f21f00b51e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.605+0000 7f21efe00640 1 -- 172.21.15.16:0/2433033914 >> 172.21.15.16:0/2433033914 conn(0x7f21f0098e30 msgr2=0x7f21f0068630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:20.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.605+0000 7f21efe00640 1 -- 172.21.15.16:0/2433033914 shutdown_connections 2024-08-17T02:46:20.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.605+0000 7f21efe00640 1 -- 172.21.15.16:0/2433033914 wait complete. 2024-08-17T02:46:20.622 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-08-17T02:46:20.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:46:20.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.773+0000 7f3971e00640 1 Processor -- start 2024-08-17T02:46:20.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.773+0000 7f3971e00640 1 -- start start 2024-08-17T02:46:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.774+0000 7f3971e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f396c155a20 0x7f396c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.774+0000 7f3971e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f396c06df00 0x7f396c0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.774+0000 7f3971e00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f396c156670 con 0x7f396c155df0 2024-08-17T02:46:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f3971e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f396c0b0350 con 0x7f396c06df00 2024-08-17T02:46:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f3971e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396c06e5e0 con 0x7f396c155a20 2024-08-17T02:46:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f3963e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f396c155df0 0x7f396c06d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57290/0 (socket says 172.21.15.16:57290) 2024-08-17T02:46:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f3963e00640 1 -- 172.21.15.16:0/4236553591 learned_addr learned my addr 172.21.15.16:0/4236553591 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f396aa00640 1 --2- 172.21.15.16:0/4236553591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f396c155a20 0x7f396c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:20.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f396b400640 1 --2- 172.21.15.16:0/4236553591 >> v2:172.21.15.142:3300/0 conn(0x7f396c06df00 0x7f396c0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f396a000640 1 -- 172.21.15.16:0/4236553591 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854160746 0 0) 0x7f396c156670 con 0x7f396c155df0 2024-08-17T02:46:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f396a000640 1 -- 172.21.15.16:0/4236553591 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39500034e0 con 0x7f396c155df0 2024-08-17T02:46:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f396b400640 1 -- 172.21.15.16:0/4236553591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f396c155a20 msgr2=0x7f396c0adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f396b400640 1 --2- 172.21.15.16:0/4236553591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f396c155a20 0x7f396c0adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f396b400640 1 -- 172.21.15.16:0/4236553591 >> v1:172.21.15.16:6789/0 conn(0x7f396c155df0 legacy=0x7f396c06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f396b400640 1 -- 172.21.15.16:0/4236553591 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396c06e8c0 con 0x7f396c06df00 2024-08-17T02:46:20.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.775+0000 7f396aa00640 1 --2- 172.21.15.16:0/4236553591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f396c155a20 0x7f396c0adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:46:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.777+0000 7f396b400640 1 --2- 172.21.15.16:0/4236553591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396c06df00 0x7f396c0ad9b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3964009e60 tx=0x7f3964009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff791f72d45b0bd server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.777+0000 7f396a000640 1 -- 172.21.15.16:0/4236553591 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964007550 con 0x7f396c06df00 2024-08-17T02:46:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.777+0000 7f396a000640 1 -- 172.21.15.16:0/4236553591 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f396402d050 con 0x7f396c06df00 2024-08-17T02:46:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.777+0000 7f396a000640 1 -- 172.21.15.16:0/4236553591 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3964007900 con 0x7f396c06df00 2024-08-17T02:46:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.777+0000 7f3971e00640 1 -- 172.21.15.16:0/4236553591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396c06df00 msgr2=0x7f396c0ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.777+0000 7f3971e00640 1 --2- 172.21.15.16:0/4236553591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396c06df00 0x7f396c0ad9b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3964009e60 tx=0x7f3964009780 comp rx=0 tx=0).stop 2024-08-17T02:46:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.778+0000 7f3971e00640 1 -- 172.21.15.16:0/4236553591 shutdown_connections 2024-08-17T02:46:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.778+0000 7f3971e00640 1 --2- 172.21.15.16:0/4236553591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396c06df00 0x7f396c0ad9b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.778+0000 7f3971e00640 1 --2- 172.21.15.16:0/4236553591 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f396c155a20 0x7f396c0adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.778+0000 7f3971e00640 1 -- 172.21.15.16:0/4236553591 >> 172.21.15.16:0/4236553591 conn(0x7f396c0cb9b0 msgr2=0x7f396c0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:20.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.778+0000 7f3971e00640 1 -- 172.21.15.16:0/4236553591 shutdown_connections 2024-08-17T02:46:20.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.778+0000 7f3971e00640 1 -- 172.21.15.16:0/4236553591 wait complete. 2024-08-17T02:46:20.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.779+0000 7f3971e00640 1 Processor -- start 2024-08-17T02:46:20.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.779+0000 7f3971e00640 1 -- start start 2024-08-17T02:46:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.780+0000 7f3971e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396c06df00 0x7f396c0acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.780+0000 7f3971e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396c155a20 0x7f396c0aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.780+0000 7f3971e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f396c155df0 0x7f396c152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.780+0000 7f3971e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c13df60 con 0x7f396c155a20 2024-08-17T02:46:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.780+0000 7f3971e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c0a8b50 con 0x7f396c06df00 2024-08-17T02:46:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.780+0000 7f3971e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f396c0a7f40 con 0x7f396c155df0 2024-08-17T02:46:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.780+0000 7f3963e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396c155a20 0x7f396c0aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.780+0000 7f396aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396c06df00 0x7f396c0acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.780+0000 7f396b400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f396c155df0 0x7f396c152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.780+0000 7f3963e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396c155a20 0x7f396c0aa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43108/0 (socket says 172.21.15.16:43108) 2024-08-17T02:46:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.780+0000 7f3963e00640 1 -- 172.21.15.16:0/122011765 learned_addr learned my addr 172.21.15.16:0/122011765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.780+0000 7f396aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396c06df00 0x7f396c0acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:53686/0 (socket says 172.21.15.16:53686) 2024-08-17T02:46:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.781+0000 7f3963e00640 1 -- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f396c155df0 msgr2=0x7f396c152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.781+0000 7f3963e00640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f396c155df0 0x7f396c152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.781+0000 7f3963e00640 1 -- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396c06df00 msgr2=0x7f396c0acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.781+0000 7f3963e00640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396c06df00 0x7f396c0acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.781+0000 7f3963e00640 1 -- 172.21.15.16:0/122011765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395c009390 con 0x7f396c155a20 2024-08-17T02:46:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.781+0000 7f396aa00640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396c06df00 0x7f396c0acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:46:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.781+0000 7f3963e00640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396c155a20 0x7f396c0aa8c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f395400c4b0 tx=0x7f3954007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.781+0000 7f3962000640 1 -- 172.21.15.16:0/122011765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3954005aa0 con 0x7f396c155a20 2024-08-17T02:46:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.781+0000 7f3962000640 1 -- 172.21.15.16:0/122011765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3954005c00 con 0x7f396c155a20 2024-08-17T02:46:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.781+0000 7f3962000640 1 -- 172.21.15.16:0/122011765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39540070d0 con 0x7f396c155a20 2024-08-17T02:46:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.781+0000 7f3971e00640 1 -- 172.21.15.16:0/122011765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3964009a20 con 0x7f396c155a20 2024-08-17T02:46:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.782+0000 7f3971e00640 1 -- 172.21.15.16:0/122011765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396c0a8500 con 0x7f396c155a20 2024-08-17T02:46:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.783+0000 7f3971e00640 1 -- 172.21.15.16:0/122011765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396c064db0 con 0x7f396c155a20 2024-08-17T02:46:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.783+0000 7f3962000640 1 -- 172.21.15.16:0/122011765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f39540074c0 con 0x7f396c155a20 2024-08-17T02:46:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.784+0000 7f3962000640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3948065180 0x7f3948067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.784+0000 7f3962000640 1 -- 172.21.15.16:0/122011765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3954081870 con 0x7f396c155a20 2024-08-17T02:46:20.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.785+0000 7f396aa00640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3948065180 0x7f3948067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:20.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.785+0000 7f396aa00640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3948065180 0x7f3948067630 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f395c011580 tx=0x7f395c0090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:20.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.788+0000 7f3962000640 1 -- 172.21.15.16:0/122011765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f395404f940 con 0x7f396c155a20 2024-08-17T02:46:20.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.972+0000 7f3971e00640 1 -- 172.21.15.16:0/122011765 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f396c1535b0 con 0x7f3948065180 2024-08-17T02:46:20.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.980+0000 7f3962000640 1 -- 172.21.15.16:0/122011765 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288953 (secure 0 0 0) 0x7f396c1535b0 con 0x7f3948065180 2024-08-17T02:46:20.985 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:46:20.987 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-17T02:46:20.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.984+0000 7f3971e00640 1 -- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3948065180 msgr2=0x7f3948067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.984+0000 7f3971e00640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3948065180 0x7f3948067630 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f395c011580 tx=0x7f395c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.984+0000 7f3971e00640 1 -- 172.21.15.16:0/122011765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396c155a20 msgr2=0x7f396c0aa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:20.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.984+0000 7f3971e00640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396c155a20 0x7f396c0aa8c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f395400c4b0 tx=0x7f3954007d60 comp rx=0 tx=0).stop 2024-08-17T02:46:20.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.984+0000 7f3971e00640 1 -- 172.21.15.16:0/122011765 shutdown_connections 2024-08-17T02:46:20.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.985+0000 7f3971e00640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3948065180 0x7f3948067630 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.985+0000 7f3971e00640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f396c155df0 0x7f396c152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.985+0000 7f3971e00640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f396c155a20 0x7f396c0aa8c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.985+0000 7f3971e00640 1 --2- 172.21.15.16:0/122011765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396c06df00 0x7f396c0acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:20.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.985+0000 7f3971e00640 1 -- 172.21.15.16:0/122011765 >> 172.21.15.16:0/122011765 conn(0x7f396c0cb9b0 msgr2=0x7f396c069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:20.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.985+0000 7f3971e00640 1 -- 172.21.15.16:0/122011765 shutdown_connections 2024-08-17T02:46:20.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:20.985+0000 7f3971e00640 1 -- 172.21.15.16:0/122011765 wait complete. 2024-08-17T02:46:21.000 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-08-17T02:46:20.250198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2399600,"kb_used_data":4064,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752575120,"statfs":{"total":773094113280,"available":770636922880,"internally_reserved":0,"allocated":4161536,"data_stored":2297156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.499790"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.798064+0000","last_change":"2024-08-17T02:46:16.780710+0000","last_active":"2024-08-17T02:46:16.798064+0000","last_peered":"2024-08-17T02:46:16.798064+0000","last_clean":"2024-08-17T02:46:16.798064+0000","last_became_active":"2024-08-17T02:46:16.780129+0000","last_became_peered":"2024-08-17T02:46:16.780129+0000","last_unstale":"2024-08-17T02:46:16.798064+0000","last_undegraded":"2024-08-17T02:46:16.798064+0000","last_fullsized":"2024-08-17T02:46:16.798064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:06:28.508134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.788411+0000","last_change":"2024-08-17T02:46:16.788411+0000","last_active":"2024-08-17T02:46:16.788411+0000","last_peered":"2024-08-17T02:46:16.788411+0000","last_clean":"2024-08-17T02:46:16.788411+0000","last_became_active":"2024-08-17T02:46:16.786270+0000","last_became_peered":"2024-08-17T02:46:16.786270+0000","last_unstale":"2024-08-17T02:46:16.788411+0000","last_undegraded":"2024-08-17T02:46:16.788411+0000","last_fullsized":"2024-08-17T02:46:16.788411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:32:01.689387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.807045+0000","last_change":"2024-08-17T02:46:16.803017+0000","last_active":"2024-08-17T02:46:16.807045+0000","last_peered":"2024-08-17T02:46:16.807045+0000","last_clean":"2024-08-17T02:46:16.807045+0000","last_became_active":"2024-08-17T02:46:16.802772+0000","last_became_peered":"2024-08-17T02:46:16.802772+0000","last_unstale":"2024-08-17T02:46:16.807045+0000","last_undegraded":"2024-08-17T02:46:16.807045+0000","last_fullsized":"2024-08-17T02:46:16.807045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:31:50.354534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.790489+0000","last_change":"2024-08-17T02:46:16.786268+0000","last_active":"2024-08-17T02:46:16.790489+0000","last_peered":"2024-08-17T02:46:16.790489+0000","last_clean":"2024-08-17T02:46:16.790489+0000","last_became_active":"2024-08-17T02:46:16.786015+0000","last_became_peered":"2024-08-17T02:46:16.786015+0000","last_unstale":"2024-08-17T02:46:16.790489+0000","last_undegraded":"2024-08-17T02:46:16.790489+0000","last_fullsized":"2024-08-17T02:46:16.790489+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:43:02.974223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.772747+0000","last_change":"2024-08-17T02:46:16.772747+0000","last_active":"2024-08-17T02:46:16.772747+0000","last_peered":"2024-08-17T02:46:16.772747+0000","last_clean":"2024-08-17T02:46:16.772747+0000","last_became_active":"2024-08-17T02:46:16.772510+0000","last_became_peered":"2024-08-17T02:46:16.772510+0000","last_unstale":"2024-08-17T02:46:16.772747+0000","last_undegraded":"2024-08-17T02:46:16.772747+0000","last_fullsized":"2024-08-17T02:46:16.772747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:13:45.059552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.789758+0000","last_change":"2024-08-17T02:46:16.785700+0000","last_active":"2024-08-17T02:46:16.789758+0000","last_peered":"2024-08-17T02:46:16.789758+0000","last_clean":"2024-08-17T02:46:16.789758+0000","last_became_active":"2024-08-17T02:46:16.785413+0000","last_became_peered":"2024-08-17T02:46:16.785413+0000","last_unstale":"2024-08-17T02:46:16.789758+0000","last_undegraded":"2024-08-17T02:46:16.789758+0000","last_fullsized":"2024-08-17T02:46:16.789758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:47:09.835864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.808530+0000","last_change":"2024-08-17T02:46:16.802806+0000","last_active":"2024-08-17T02:46:16.808530+0000","last_peered":"2024-08-17T02:46:16.808530+0000","last_clean":"2024-08-17T02:46:16.808530+0000","last_became_active":"2024-08-17T02:46:16.802495+0000","last_became_peered":"2024-08-17T02:46:16.802495+0000","last_unstale":"2024-08-17T02:46:16.808530+0000","last_undegraded":"2024-08-17T02:46:16.808530+0000","last_fullsized":"2024-08-17T02:46:16.808530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:43.890584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.789538+0000","last_change":"2024-08-17T02:46:16.779804+0000","last_active":"2024-08-17T02:46:16.789538+0000","last_peered":"2024-08-17T02:46:16.789538+0000","last_clean":"2024-08-17T02:46:16.789538+0000","last_became_active":"2024-08-17T02:46:16.779553+0000","last_became_peered":"2024-08-17T02:46:16.779553+0000","last_unstale":"2024-08-17T02:46:16.789538+0000","last_undegraded":"2024-08-17T02:46:16.789538+0000","last_fullsized":"2024-08-17T02:46:16.789538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:44:22.048521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.796842+0000","last_change":"2024-08-17T02:46:16.790616+0000","last_active":"2024-08-17T02:46:16.796842+0000","last_peered":"2024-08-17T02:46:16.796842+0000","last_clean":"2024-08-17T02:46:16.796842+0000","last_became_active":"2024-08-17T02:46:16.790362+0000","last_became_peered":"2024-08-17T02:46:16.790362+0000","last_unstale":"2024-08-17T02:46:16.796842+0000","last_undegraded":"2024-08-17T02:46:16.796842+0000","last_fullsized":"2024-08-17T02:46:16.796842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:51:11.224931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.797531+0000","last_change":"2024-08-17T02:46:16.792038+0000","last_active":"2024-08-17T02:46:16.797531+0000","last_peered":"2024-08-17T02:46:16.797531+0000","last_clean":"2024-08-17T02:46:16.797531+0000","last_became_active":"2024-08-17T02:46:16.791769+0000","last_became_peered":"2024-08-17T02:46:16.791769+0000","last_unstale":"2024-08-17T02:46:16.797531+0000","last_undegraded":"2024-08-17T02:46:16.797531+0000","last_fullsized":"2024-08-17T02:46:16.797531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:44:38.564320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.786660+0000","last_change":"2024-08-17T02:46:16.786660+0000","last_active":"2024-08-17T02:46:16.786660+0000","last_peered":"2024-08-17T02:46:16.786660+0000","last_clean":"2024-08-17T02:46:16.786660+0000","last_became_active":"2024-08-17T02:46:16.786057+0000","last_became_peered":"2024-08-17T02:46:16.786057+0000","last_unstale":"2024-08-17T02:46:16.786660+0000","last_undegraded":"2024-08-17T02:46:16.786660+0000","last_fullsized":"2024-08-17T02:46:16.786660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:48:48.175556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.799505+0000","last_change":"2024-08-17T02:46:16.790194+0000","last_active":"2024-08-17T02:46:16.799505+0000","last_peered":"2024-08-17T02:46:16.799505+0000","last_clean":"2024-08-17T02:46:16.799505+0000","last_became_active":"2024-08-17T02:46:16.789942+0000","last_became_peered":"2024-08-17T02:46:16.789942+0000","last_unstale":"2024-08-17T02:46:16.799505+0000","last_undegraded":"2024-08-17T02:46:16.799505+0000","last_fullsized":"2024-08-17T02:46:16.799505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:43:14.093453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.781804+0000","last_change":"2024-08-17T02:46:16.781804+0000","last_active":"2024-08-17T02:46:16.781804+0000","last_peered":"2024-08-17T02:46:16.781804+0000","last_clean":"2024-08-17T02:46:16.781804+0000","last_became_active":"2024-08-17T02:46:16.781527+0000","last_became_peered":"2024-08-17T02:46:16.781527+0000","last_unstale":"2024-08-17T02:46:16.781804+0000","last_undegraded":"2024-08-17T02:46:16.781804+0000","last_fullsized":"2024-08-17T02:46:16.781804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:02:07.541310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.785745+0000","last_change":"2024-08-17T02:46:16.780339+0000","last_active":"2024-08-17T02:46:16.785745+0000","last_peered":"2024-08-17T02:46:16.785745+0000","last_clean":"2024-08-17T02:46:16.785745+0000","last_became_active":"2024-08-17T02:46:16.779588+0000","last_became_peered":"2024-08-17T02:46:16.779588+0000","last_unstale":"2024-08-17T02:46:16.785745+0000","last_undegraded":"2024-08-17T02:46:16.785745+0000","last_fullsized":"2024-08-17T02:46:16.785745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:19:59.271493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.780568+0000","last_change":"2024-08-17T02:46:16.780568+0000","last_active":"2024-08-17T02:46:16.780568+0000","last_peered":"2024-08-17T02:46:16.780568+0000","last_clean":"2024-08-17T02:46:16.780568+0000","last_became_active":"2024-08-17T02:46:16.780268+0000","last_became_peered":"2024-08-17T02:46:16.780268+0000","last_unstale":"2024-08-17T02:46:16.780568+0000","last_undegraded":"2024-08-17T02:46:16.780568+0000","last_fullsized":"2024-08-17T02:46:16.780568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:13:45.059552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.783449+0000","last_change":"2024-08-17T02:46:16.783449+0000","last_active":"2024-08-17T02:46:16.783449+0000","last_peered":"2024-08-17T02:46:16.783449+0000","last_clean":"2024-08-17T02:46:16.783449+0000","last_became_active":"2024-08-17T02:46:16.782513+0000","last_became_peered":"2024-08-17T02:46:16.782513+0000","last_unstale":"2024-08-17T02:46:16.783449+0000","last_undegraded":"2024-08-17T02:46:16.783449+0000","last_fullsized":"2024-08-17T02:46:16.783449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:59:23.376039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.783117+0000","last_change":"2024-08-17T02:46:16.783117+0000","last_active":"2024-08-17T02:46:16.783117+0000","last_peered":"2024-08-17T02:46:16.783117+0000","last_clean":"2024-08-17T02:46:16.783117+0000","last_became_active":"2024-08-17T02:46:16.781944+0000","last_became_peered":"2024-08-17T02:46:16.781944+0000","last_unstale":"2024-08-17T02:46:16.783117+0000","last_undegraded":"2024-08-17T02:46:16.783117+0000","last_fullsized":"2024-08-17T02:46:16.783117+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:21:47.106619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.786286+0000","last_change":"2024-08-17T02:46:16.783330+0000","last_active":"2024-08-17T02:46:16.786286+0000","last_peered":"2024-08-17T02:46:16.786286+0000","last_clean":"2024-08-17T02:46:16.786286+0000","last_became_active":"2024-08-17T02:46:16.782741+0000","last_became_peered":"2024-08-17T02:46:16.782741+0000","last_unstale":"2024-08-17T02:46:16.786286+0000","last_undegraded":"2024-08-17T02:46:16.786286+0000","last_fullsized":"2024-08-17T02:46:16.786286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:03:00.932909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.792670+0000","last_change":"2024-08-17T02:46:16.791359+0000","last_active":"2024-08-17T02:46:16.792670+0000","last_peered":"2024-08-17T02:46:16.792670+0000","last_clean":"2024-08-17T02:46:16.792670+0000","last_became_active":"2024-08-17T02:46:16.791083+0000","last_became_peered":"2024-08-17T02:46:16.791083+0000","last_unstale":"2024-08-17T02:46:16.792670+0000","last_undegraded":"2024-08-17T02:46:16.792670+0000","last_fullsized":"2024-08-17T02:46:16.792670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:43:35.816603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.778355+0000","last_change":"2024-08-17T02:46:16.778355+0000","last_active":"2024-08-17T02:46:16.778355+0000","last_peered":"2024-08-17T02:46:16.778355+0000","last_clean":"2024-08-17T02:46:16.778355+0000","last_became_active":"2024-08-17T02:46:16.778072+0000","last_became_peered":"2024-08-17T02:46:16.778072+0000","last_unstale":"2024-08-17T02:46:16.778355+0000","last_undegraded":"2024-08-17T02:46:16.778355+0000","last_fullsized":"2024-08-17T02:46:16.778355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:23:25.110064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.775559+0000","last_change":"2024-08-17T02:46:16.775559+0000","last_active":"2024-08-17T02:46:16.775559+0000","last_peered":"2024-08-17T02:46:16.775559+0000","last_clean":"2024-08-17T02:46:16.775559+0000","last_became_active":"2024-08-17T02:46:16.775281+0000","last_became_peered":"2024-08-17T02:46:16.775281+0000","last_unstale":"2024-08-17T02:46:16.775559+0000","last_undegraded":"2024-08-17T02:46:16.775559+0000","last_fullsized":"2024-08-17T02:46:16.775559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:38.566217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.785553+0000","last_change":"2024-08-17T02:46:16.777922+0000","last_active":"2024-08-17T02:46:16.785553+0000","last_peered":"2024-08-17T02:46:16.785553+0000","last_clean":"2024-08-17T02:46:16.785553+0000","last_became_active":"2024-08-17T02:46:16.777695+0000","last_became_peered":"2024-08-17T02:46:16.777695+0000","last_unstale":"2024-08-17T02:46:16.785553+0000","last_undegraded":"2024-08-17T02:46:16.785553+0000","last_fullsized":"2024-08-17T02:46:16.785553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:19:13.983272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.807299+0000","last_change":"2024-08-17T02:46:16.807299+0000","last_active":"2024-08-17T02:46:16.807299+0000","last_peered":"2024-08-17T02:46:16.807299+0000","last_clean":"2024-08-17T02:46:16.807299+0000","last_became_active":"2024-08-17T02:46:16.807104+0000","last_became_peered":"2024-08-17T02:46:16.807104+0000","last_unstale":"2024-08-17T02:46:16.807299+0000","last_undegraded":"2024-08-17T02:46:16.807299+0000","last_fullsized":"2024-08-17T02:46:16.807299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:34:45.612431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.804741+0000","last_change":"2024-08-17T02:46:16.790102+0000","last_active":"2024-08-17T02:46:16.804741+0000","last_peered":"2024-08-17T02:46:16.804741+0000","last_clean":"2024-08-17T02:46:16.804741+0000","last_became_active":"2024-08-17T02:46:16.789831+0000","last_became_peered":"2024-08-17T02:46:16.789831+0000","last_unstale":"2024-08-17T02:46:16.804741+0000","last_undegraded":"2024-08-17T02:46:16.804741+0000","last_fullsized":"2024-08-17T02:46:16.804741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:31:24.953450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.787399+0000","last_change":"2024-08-17T02:46:16.783028+0000","last_active":"2024-08-17T02:46:16.787399+0000","last_peered":"2024-08-17T02:46:16.787399+0000","last_clean":"2024-08-17T02:46:16.787399+0000","last_became_active":"2024-08-17T02:46:16.782595+0000","last_became_peered":"2024-08-17T02:46:16.782595+0000","last_unstale":"2024-08-17T02:46:16.787399+0000","last_undegraded":"2024-08-17T02:46:16.787399+0000","last_fullsized":"2024-08-17T02:46:16.787399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:25:19.636862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.804818+0000","last_change":"2024-08-17T02:46:16.801306+0000","last_active":"2024-08-17T02:46:16.804818+0000","last_peered":"2024-08-17T02:46:16.804818+0000","last_clean":"2024-08-17T02:46:16.804818+0000","last_became_active":"2024-08-17T02:46:16.800998+0000","last_became_peered":"2024-08-17T02:46:16.800998+0000","last_unstale":"2024-08-17T02:46:16.804818+0000","last_undegraded":"2024-08-17T02:46:16.804818+0000","last_fullsized":"2024-08-17T02:46:16.804818+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:44:22.048521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.773451+0000","last_change":"2024-08-17T02:46:16.773451+0000","last_active":"2024-08-17T02:46:16.773451+0000","last_peered":"2024-08-17T02:46:16.773451+0000","last_clean":"2024-08-17T02:46:16.773451+0000","last_became_active":"2024-08-17T02:46:16.773189+0000","last_became_peered":"2024-08-17T02:46:16.773189+0000","last_unstale":"2024-08-17T02:46:16.773451+0000","last_undegraded":"2024-08-17T02:46:16.773451+0000","last_fullsized":"2024-08-17T02:46:16.773451+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:39:22.034306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.805135+0000","last_change":"2024-08-17T02:46:16.797624+0000","last_active":"2024-08-17T02:46:16.805135+0000","last_peered":"2024-08-17T02:46:16.805135+0000","last_clean":"2024-08-17T02:46:16.805135+0000","last_became_active":"2024-08-17T02:46:16.790772+0000","last_became_peered":"2024-08-17T02:46:16.790772+0000","last_unstale":"2024-08-17T02:46:16.805135+0000","last_undegraded":"2024-08-17T02:46:16.805135+0000","last_fullsized":"2024-08-17T02:46:16.805135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:19:59.271493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.784253+0000","last_change":"2024-08-17T02:46:16.784253+0000","last_active":"2024-08-17T02:46:16.784253+0000","last_peered":"2024-08-17T02:46:16.784253+0000","last_clean":"2024-08-17T02:46:16.784253+0000","last_became_active":"2024-08-17T02:46:16.782422+0000","last_became_peered":"2024-08-17T02:46:16.782422+0000","last_unstale":"2024-08-17T02:46:16.784253+0000","last_undegraded":"2024-08-17T02:46:16.784253+0000","last_fullsized":"2024-08-17T02:46:16.784253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:44:22.048521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.778183+0000","last_change":"2024-08-17T02:46:16.778183+0000","last_active":"2024-08-17T02:46:16.778183+0000","last_peered":"2024-08-17T02:46:16.778183+0000","last_clean":"2024-08-17T02:46:16.778183+0000","last_became_active":"2024-08-17T02:46:16.777830+0000","last_became_peered":"2024-08-17T02:46:16.777830+0000","last_unstale":"2024-08-17T02:46:16.778183+0000","last_undegraded":"2024-08-17T02:46:16.778183+0000","last_fullsized":"2024-08-17T02:46:16.778183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:39:22.034306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.797835+0000","last_change":"2024-08-17T02:46:16.785642+0000","last_active":"2024-08-17T02:46:16.797835+0000","last_peered":"2024-08-17T02:46:16.797835+0000","last_clean":"2024-08-17T02:46:16.797835+0000","last_became_active":"2024-08-17T02:46:16.785329+0000","last_became_peered":"2024-08-17T02:46:16.785329+0000","last_unstale":"2024-08-17T02:46:16.797835+0000","last_undegraded":"2024-08-17T02:46:16.797835+0000","last_fullsized":"2024-08-17T02:46:16.797835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:59:23.376039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.793188+0000","last_change":"2024-08-17T02:46:16.793188+0000","last_active":"2024-08-17T02:46:16.793188+0000","last_peered":"2024-08-17T02:46:16.793188+0000","last_clean":"2024-08-17T02:46:16.793188+0000","last_became_active":"2024-08-17T02:46:16.792764+0000","last_became_peered":"2024-08-17T02:46:16.792764+0000","last_unstale":"2024-08-17T02:46:16.793188+0000","last_undegraded":"2024-08-17T02:46:16.793188+0000","last_fullsized":"2024-08-17T02:46:16.793188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:45:18.517491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.779163+0000","last_change":"2024-08-17T02:46:16.779163+0000","last_active":"2024-08-17T02:46:16.779163+0000","last_peered":"2024-08-17T02:46:16.779163+0000","last_clean":"2024-08-17T02:46:16.779163+0000","last_became_active":"2024-08-17T02:46:16.778884+0000","last_became_peered":"2024-08-17T02:46:16.778884+0000","last_unstale":"2024-08-17T02:46:16.779163+0000","last_undegraded":"2024-08-17T02:46:16.779163+0000","last_fullsized":"2024-08-17T02:46:16.779163+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:38:11.703986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.803561+0000","last_change":"2024-08-17T02:46:15.778364+0000","last_active":"2024-08-17T02:46:16.803561+0000","last_peered":"2024-08-17T02:46:16.803561+0000","last_clean":"2024-08-17T02:46:16.803561+0000","last_became_active":"2024-08-17T02:46:15.778053+0000","last_became_peered":"2024-08-17T02:46:15.778053+0000","last_unstale":"2024-08-17T02:46:16.803561+0000","last_undegraded":"2024-08-17T02:46:16.803561+0000","last_fullsized":"2024-08-17T02:46:16.803561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:59:22.369433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.798087+0000","last_change":"2024-08-17T02:46:16.780991+0000","last_active":"2024-08-17T02:46:16.798087+0000","last_peered":"2024-08-17T02:46:16.798087+0000","last_clean":"2024-08-17T02:46:16.798087+0000","last_became_active":"2024-08-17T02:46:16.780479+0000","last_became_peered":"2024-08-17T02:46:16.780479+0000","last_unstale":"2024-08-17T02:46:16.798087+0000","last_undegraded":"2024-08-17T02:46:16.798087+0000","last_fullsized":"2024-08-17T02:46:16.798087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:44:38.564320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.807436+0000","last_change":"2024-08-17T02:46:15.779781+0000","last_active":"2024-08-17T02:46:16.807436+0000","last_peered":"2024-08-17T02:46:16.807436+0000","last_clean":"2024-08-17T02:46:16.807436+0000","last_became_active":"2024-08-17T02:46:15.779599+0000","last_became_peered":"2024-08-17T02:46:15.779599+0000","last_unstale":"2024-08-17T02:46:16.807436+0000","last_undegraded":"2024-08-17T02:46:16.807436+0000","last_fullsized":"2024-08-17T02:46:16.807436+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:32:57.038113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.808589+0000","last_change":"2024-08-17T02:46:16.799787+0000","last_active":"2024-08-17T02:46:16.808589+0000","last_peered":"2024-08-17T02:46:16.808589+0000","last_clean":"2024-08-17T02:46:16.808589+0000","last_became_active":"2024-08-17T02:46:16.790378+0000","last_became_peered":"2024-08-17T02:46:16.790378+0000","last_unstale":"2024-08-17T02:46:16.808589+0000","last_undegraded":"2024-08-17T02:46:16.808589+0000","last_fullsized":"2024-08-17T02:46:16.808589+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:24:25.550110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.800456+0000","last_change":"2024-08-17T02:46:15.784541+0000","last_active":"2024-08-17T02:46:16.800456+0000","last_peered":"2024-08-17T02:46:16.800456+0000","last_clean":"2024-08-17T02:46:16.800456+0000","last_became_active":"2024-08-17T02:46:15.784063+0000","last_became_peered":"2024-08-17T02:46:15.784063+0000","last_unstale":"2024-08-17T02:46:16.800456+0000","last_undegraded":"2024-08-17T02:46:16.800456+0000","last_fullsized":"2024-08-17T02:46:16.800456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:31:13.415009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.790058+0000","last_change":"2024-08-17T02:46:16.785115+0000","last_active":"2024-08-17T02:46:16.790058+0000","last_peered":"2024-08-17T02:46:16.790058+0000","last_clean":"2024-08-17T02:46:16.790058+0000","last_became_active":"2024-08-17T02:46:16.784009+0000","last_became_peered":"2024-08-17T02:46:16.784009+0000","last_unstale":"2024-08-17T02:46:16.790058+0000","last_undegraded":"2024-08-17T02:46:16.790058+0000","last_fullsized":"2024-08-17T02:46:16.790058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:43.890584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.805211+0000","last_change":"2024-08-17T02:46:15.786518+0000","last_active":"2024-08-17T02:46:16.805211+0000","last_peered":"2024-08-17T02:46:16.805211+0000","last_clean":"2024-08-17T02:46:16.805211+0000","last_became_active":"2024-08-17T02:46:15.785432+0000","last_became_peered":"2024-08-17T02:46:15.785432+0000","last_unstale":"2024-08-17T02:46:16.805211+0000","last_undegraded":"2024-08-17T02:46:16.805211+0000","last_fullsized":"2024-08-17T02:46:16.805211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:43:34.809998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.800722+0000","last_change":"2024-08-17T02:46:16.795299+0000","last_active":"2024-08-17T02:46:16.800722+0000","last_peered":"2024-08-17T02:46:16.800722+0000","last_clean":"2024-08-17T02:46:16.800722+0000","last_became_active":"2024-08-17T02:46:16.787463+0000","last_became_peered":"2024-08-17T02:46:16.787463+0000","last_unstale":"2024-08-17T02:46:16.800722+0000","last_undegraded":"2024-08-17T02:46:16.800722+0000","last_fullsized":"2024-08-17T02:46:16.800722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:04.740084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.812938+0000","last_change":"2024-08-17T02:46:15.776329+0000","last_active":"2024-08-17T02:46:16.812938+0000","last_peered":"2024-08-17T02:46:16.812938+0000","last_clean":"2024-08-17T02:46:16.812938+0000","last_became_active":"2024-08-17T02:46:15.774914+0000","last_became_peered":"2024-08-17T02:46:15.774914+0000","last_unstale":"2024-08-17T02:46:16.812938+0000","last_undegraded":"2024-08-17T02:46:16.812938+0000","last_fullsized":"2024-08-17T02:46:16.812938+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:37:24.295411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.794683+0000","last_change":"2024-08-17T02:46:16.784692+0000","last_active":"2024-08-17T02:46:16.794683+0000","last_peered":"2024-08-17T02:46:16.794683+0000","last_clean":"2024-08-17T02:46:16.794683+0000","last_became_active":"2024-08-17T02:46:16.784443+0000","last_became_peered":"2024-08-17T02:46:16.784443+0000","last_unstale":"2024-08-17T02:46:16.794683+0000","last_undegraded":"2024-08-17T02:46:16.794683+0000","last_fullsized":"2024-08-17T02:46:16.794683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:38.566217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.810145+0000","last_change":"2024-08-17T02:46:15.783239+0000","last_active":"2024-08-17T02:46:16.810145+0000","last_peered":"2024-08-17T02:46:16.810145+0000","last_clean":"2024-08-17T02:46:16.810145+0000","last_became_active":"2024-08-17T02:46:15.782992+0000","last_became_peered":"2024-08-17T02:46:15.782992+0000","last_unstale":"2024-08-17T02:46:16.810145+0000","last_undegraded":"2024-08-17T02:46:16.810145+0000","last_fullsized":"2024-08-17T02:46:16.810145+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:32:57.038113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.796815+0000","last_change":"2024-08-17T02:46:16.794186+0000","last_active":"2024-08-17T02:46:16.796815+0000","last_peered":"2024-08-17T02:46:16.796815+0000","last_clean":"2024-08-17T02:46:16.796815+0000","last_became_active":"2024-08-17T02:46:16.793863+0000","last_became_peered":"2024-08-17T02:46:16.793863+0000","last_unstale":"2024-08-17T02:46:16.796815+0000","last_undegraded":"2024-08-17T02:46:16.796815+0000","last_fullsized":"2024-08-17T02:46:16.796815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:36:20.309762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.776653+0000","last_change":"2024-08-17T02:46:15.785254+0000","last_active":"2024-08-17T02:46:16.776653+0000","last_peered":"2024-08-17T02:46:16.776653+0000","last_clean":"2024-08-17T02:46:16.776653+0000","last_became_active":"2024-08-17T02:46:15.784911+0000","last_became_peered":"2024-08-17T02:46:15.784911+0000","last_unstale":"2024-08-17T02:46:16.776653+0000","last_undegraded":"2024-08-17T02:46:16.776653+0000","last_fullsized":"2024-08-17T02:46:16.776653+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:17:51.619358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.785567+0000","last_change":"2024-08-17T02:46:16.781047+0000","last_active":"2024-08-17T02:46:16.785567+0000","last_peered":"2024-08-17T02:46:16.785567+0000","last_clean":"2024-08-17T02:46:16.785567+0000","last_became_active":"2024-08-17T02:46:16.780772+0000","last_became_peered":"2024-08-17T02:46:16.780772+0000","last_unstale":"2024-08-17T02:46:16.785567+0000","last_undegraded":"2024-08-17T02:46:16.785567+0000","last_fullsized":"2024-08-17T02:46:16.785567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:16:59.558851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.766983+0000","last_change":"2024-08-17T02:46:15.779695+0000","last_active":"2024-08-17T02:46:16.766983+0000","last_peered":"2024-08-17T02:46:16.766983+0000","last_clean":"2024-08-17T02:46:16.766983+0000","last_became_active":"2024-08-17T02:46:15.779452+0000","last_became_peered":"2024-08-17T02:46:15.779452+0000","last_unstale":"2024-08-17T02:46:16.766983+0000","last_undegraded":"2024-08-17T02:46:16.766983+0000","last_fullsized":"2024-08-17T02:46:16.766983+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:31:49.347928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.775032+0000","last_change":"2024-08-17T02:46:16.775032+0000","last_active":"2024-08-17T02:46:16.775032+0000","last_peered":"2024-08-17T02:46:16.775032+0000","last_clean":"2024-08-17T02:46:16.775032+0000","last_became_active":"2024-08-17T02:46:16.774773+0000","last_became_peered":"2024-08-17T02:46:16.774773+0000","last_unstale":"2024-08-17T02:46:16.775032+0000","last_undegraded":"2024-08-17T02:46:16.775032+0000","last_fullsized":"2024-08-17T02:46:16.775032+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:02:03.656421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.786914+0000","last_change":"2024-08-17T02:46:15.782088+0000","last_active":"2024-08-17T02:46:15.786914+0000","last_peered":"2024-08-17T02:46:15.786914+0000","last_clean":"2024-08-17T02:46:15.786914+0000","last_became_active":"2024-08-17T02:46:15.781896+0000","last_became_peered":"2024-08-17T02:46:15.781896+0000","last_unstale":"2024-08-17T02:46:15.786914+0000","last_undegraded":"2024-08-17T02:46:15.786914+0000","last_fullsized":"2024-08-17T02:46:15.786914+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:43:34.809998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.787380+0000","last_change":"2024-08-17T02:46:16.782862+0000","last_active":"2024-08-17T02:46:16.787380+0000","last_peered":"2024-08-17T02:46:16.787380+0000","last_clean":"2024-08-17T02:46:16.787380+0000","last_became_active":"2024-08-17T02:46:16.782332+0000","last_became_peered":"2024-08-17T02:46:16.782332+0000","last_unstale":"2024-08-17T02:46:16.787380+0000","last_undegraded":"2024-08-17T02:46:16.787380+0000","last_fullsized":"2024-08-17T02:46:16.787380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:38.566217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.780347+0000","last_change":"2024-08-17T02:46:15.774480+0000","last_active":"2024-08-17T02:46:15.780347+0000","last_peered":"2024-08-17T02:46:15.780347+0000","last_clean":"2024-08-17T02:46:15.780347+0000","last_became_active":"2024-08-17T02:46:15.774193+0000","last_became_peered":"2024-08-17T02:46:15.774193+0000","last_unstale":"2024-08-17T02:46:15.780347+0000","last_undegraded":"2024-08-17T02:46:15.780347+0000","last_fullsized":"2024-08-17T02:46:15.780347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:35:16.966461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.790910+0000","last_change":"2024-08-17T02:46:16.787330+0000","last_active":"2024-08-17T02:46:16.790910+0000","last_peered":"2024-08-17T02:46:16.790910+0000","last_clean":"2024-08-17T02:46:16.790910+0000","last_became_active":"2024-08-17T02:46:16.787060+0000","last_became_peered":"2024-08-17T02:46:16.787060+0000","last_unstale":"2024-08-17T02:46:16.790910+0000","last_undegraded":"2024-08-17T02:46:16.790910+0000","last_fullsized":"2024-08-17T02:46:16.790910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:19:13.983272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.802472+0000","last_change":"2024-08-17T02:46:15.778373+0000","last_active":"2024-08-17T02:46:16.802472+0000","last_peered":"2024-08-17T02:46:16.802472+0000","last_clean":"2024-08-17T02:46:16.802472+0000","last_became_active":"2024-08-17T02:46:15.778127+0000","last_became_peered":"2024-08-17T02:46:15.778127+0000","last_unstale":"2024-08-17T02:46:16.802472+0000","last_undegraded":"2024-08-17T02:46:16.802472+0000","last_fullsized":"2024-08-17T02:46:16.802472+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:06:27.501528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.786778+0000","last_change":"2024-08-17T02:46:16.786778+0000","last_active":"2024-08-17T02:46:16.786778+0000","last_peered":"2024-08-17T02:46:16.786778+0000","last_clean":"2024-08-17T02:46:16.786778+0000","last_became_active":"2024-08-17T02:46:16.786393+0000","last_became_peered":"2024-08-17T02:46:16.786393+0000","last_unstale":"2024-08-17T02:46:16.786778+0000","last_undegraded":"2024-08-17T02:46:16.786778+0000","last_fullsized":"2024-08-17T02:46:16.786778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:02:15.765459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.805328+0000","last_change":"2024-08-17T02:46:15.785147+0000","last_active":"2024-08-17T02:46:16.805328+0000","last_peered":"2024-08-17T02:46:16.805328+0000","last_clean":"2024-08-17T02:46:16.805328+0000","last_became_active":"2024-08-17T02:46:15.784855+0000","last_became_peered":"2024-08-17T02:46:15.784855+0000","last_unstale":"2024-08-17T02:46:16.805328+0000","last_undegraded":"2024-08-17T02:46:16.805328+0000","last_fullsized":"2024-08-17T02:46:16.805328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:45:56.402053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.789376+0000","last_change":"2024-08-17T02:46:16.789376+0000","last_active":"2024-08-17T02:46:16.789376+0000","last_peered":"2024-08-17T02:46:16.789376+0000","last_clean":"2024-08-17T02:46:16.789376+0000","last_became_active":"2024-08-17T02:46:16.786740+0000","last_became_peered":"2024-08-17T02:46:16.786740+0000","last_unstale":"2024-08-17T02:46:16.789376+0000","last_undegraded":"2024-08-17T02:46:16.789376+0000","last_fullsized":"2024-08-17T02:46:16.789376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:36:44.763322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.810160+0000","last_change":"2024-08-17T02:46:15.776876+0000","last_active":"2024-08-17T02:46:16.810160+0000","last_peered":"2024-08-17T02:46:16.810160+0000","last_clean":"2024-08-17T02:46:16.810160+0000","last_became_active":"2024-08-17T02:46:15.775681+0000","last_became_peered":"2024-08-17T02:46:15.775681+0000","last_unstale":"2024-08-17T02:46:16.810160+0000","last_undegraded":"2024-08-17T02:46:16.810160+0000","last_fullsized":"2024-08-17T02:46:16.810160+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:02:59.926303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.788132+0000","last_change":"2024-08-17T02:46:16.784377+0000","last_active":"2024-08-17T02:46:16.788132+0000","last_peered":"2024-08-17T02:46:16.788132+0000","last_clean":"2024-08-17T02:46:16.788132+0000","last_became_active":"2024-08-17T02:46:16.784056+0000","last_became_peered":"2024-08-17T02:46:16.784056+0000","last_unstale":"2024-08-17T02:46:16.788132+0000","last_undegraded":"2024-08-17T02:46:16.788132+0000","last_fullsized":"2024-08-17T02:46:16.788132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:31:24.953450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.777168+0000","last_change":"2024-08-17T02:46:15.778765+0000","last_active":"2024-08-17T02:46:16.777168+0000","last_peered":"2024-08-17T02:46:16.777168+0000","last_clean":"2024-08-17T02:46:16.777168+0000","last_became_active":"2024-08-17T02:46:15.778410+0000","last_became_peered":"2024-08-17T02:46:15.778410+0000","last_unstale":"2024-08-17T02:46:16.777168+0000","last_undegraded":"2024-08-17T02:46:16.777168+0000","last_fullsized":"2024-08-17T02:46:16.777168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:02:59.926303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.809172+0000","last_change":"2024-08-17T02:46:16.808197+0000","last_active":"2024-08-17T02:46:16.809172+0000","last_peered":"2024-08-17T02:46:16.809172+0000","last_clean":"2024-08-17T02:46:16.809172+0000","last_became_active":"2024-08-17T02:46:16.804880+0000","last_became_peered":"2024-08-17T02:46:16.804880+0000","last_unstale":"2024-08-17T02:46:16.809172+0000","last_undegraded":"2024-08-17T02:46:16.809172+0000","last_fullsized":"2024-08-17T02:46:16.809172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:34:26.609150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.776979+0000","last_change":"2024-08-17T02:46:15.780068+0000","last_active":"2024-08-17T02:46:16.776979+0000","last_peered":"2024-08-17T02:46:16.776979+0000","last_clean":"2024-08-17T02:46:16.776979+0000","last_became_active":"2024-08-17T02:46:15.778500+0000","last_became_peered":"2024-08-17T02:46:15.778500+0000","last_unstale":"2024-08-17T02:46:16.776979+0000","last_undegraded":"2024-08-17T02:46:16.776979+0000","last_fullsized":"2024-08-17T02:46:16.776979+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:51:10.218325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.781865+0000","last_change":"2024-08-17T02:46:16.781865+0000","last_active":"2024-08-17T02:46:16.781865+0000","last_peered":"2024-08-17T02:46:16.781865+0000","last_clean":"2024-08-17T02:46:16.781865+0000","last_became_active":"2024-08-17T02:46:16.778795+0000","last_became_peered":"2024-08-17T02:46:16.778795+0000","last_unstale":"2024-08-17T02:46:16.781865+0000","last_undegraded":"2024-08-17T02:46:16.781865+0000","last_fullsized":"2024-08-17T02:46:16.781865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:34:20.741425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.803935+0000","last_change":"2024-08-17T02:46:15.785957+0000","last_active":"2024-08-17T02:46:16.803935+0000","last_peered":"2024-08-17T02:46:16.803935+0000","last_clean":"2024-08-17T02:46:16.803935+0000","last_became_active":"2024-08-17T02:46:15.785692+0000","last_became_peered":"2024-08-17T02:46:15.785692+0000","last_unstale":"2024-08-17T02:46:16.803935+0000","last_undegraded":"2024-08-17T02:46:16.803935+0000","last_fullsized":"2024-08-17T02:46:16.803935+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:32:43.979148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.786826+0000","last_change":"2024-08-17T02:46:16.786826+0000","last_active":"2024-08-17T02:46:16.786826+0000","last_peered":"2024-08-17T02:46:16.786826+0000","last_clean":"2024-08-17T02:46:16.786826+0000","last_became_active":"2024-08-17T02:46:16.785671+0000","last_became_peered":"2024-08-17T02:46:16.785671+0000","last_unstale":"2024-08-17T02:46:16.786826+0000","last_undegraded":"2024-08-17T02:46:16.786826+0000","last_fullsized":"2024-08-17T02:46:16.786826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:53:59.611294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.802779+0000","last_change":"2024-08-17T02:46:15.784448+0000","last_active":"2024-08-17T02:46:16.802779+0000","last_peered":"2024-08-17T02:46:16.802779+0000","last_clean":"2024-08-17T02:46:16.802779+0000","last_became_active":"2024-08-17T02:46:15.784156+0000","last_became_peered":"2024-08-17T02:46:15.784156+0000","last_unstale":"2024-08-17T02:46:16.802779+0000","last_undegraded":"2024-08-17T02:46:16.802779+0000","last_fullsized":"2024-08-17T02:46:16.802779+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:27:14.740630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.806807+0000","last_change":"2024-08-17T02:46:16.796802+0000","last_active":"2024-08-17T02:46:16.806807+0000","last_peered":"2024-08-17T02:46:16.806807+0000","last_clean":"2024-08-17T02:46:16.806807+0000","last_became_active":"2024-08-17T02:46:16.796547+0000","last_became_peered":"2024-08-17T02:46:16.796547+0000","last_unstale":"2024-08-17T02:46:16.806807+0000","last_undegraded":"2024-08-17T02:46:16.806807+0000","last_fullsized":"2024-08-17T02:46:16.806807+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:47:09.835864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.808984+0000","last_change":"2024-08-17T02:46:15.776482+0000","last_active":"2024-08-17T02:46:16.808984+0000","last_peered":"2024-08-17T02:46:16.808984+0000","last_clean":"2024-08-17T02:46:16.808984+0000","last_became_active":"2024-08-17T02:46:15.775200+0000","last_became_peered":"2024-08-17T02:46:15.775200+0000","last_unstale":"2024-08-17T02:46:16.808984+0000","last_undegraded":"2024-08-17T02:46:16.808984+0000","last_fullsized":"2024-08-17T02:46:16.808984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:47:08.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.808041+0000","last_change":"2024-08-17T02:46:16.808041+0000","last_active":"2024-08-17T02:46:16.808041+0000","last_peered":"2024-08-17T02:46:16.808041+0000","last_clean":"2024-08-17T02:46:16.808041+0000","last_became_active":"2024-08-17T02:46:16.807798+0000","last_became_peered":"2024-08-17T02:46:16.807798+0000","last_unstale":"2024-08-17T02:46:16.808041+0000","last_undegraded":"2024-08-17T02:46:16.808041+0000","last_fullsized":"2024-08-17T02:46:16.808041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:59:23.376039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.772244+0000","last_change":"2024-08-17T02:46:15.786125+0000","last_active":"2024-08-17T02:46:16.772244+0000","last_peered":"2024-08-17T02:46:16.772244+0000","last_clean":"2024-08-17T02:46:16.772244+0000","last_became_active":"2024-08-17T02:46:15.785926+0000","last_became_peered":"2024-08-17T02:46:15.785926+0000","last_unstale":"2024-08-17T02:46:16.772244+0000","last_undegraded":"2024-08-17T02:46:16.772244+0000","last_fullsized":"2024-08-17T02:46:16.772244+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:44:37.557714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.786226+0000","last_change":"2024-08-17T02:46:16.782084+0000","last_active":"2024-08-17T02:46:16.786226+0000","last_peered":"2024-08-17T02:46:16.786226+0000","last_clean":"2024-08-17T02:46:16.786226+0000","last_became_active":"2024-08-17T02:46:16.781650+0000","last_became_peered":"2024-08-17T02:46:16.781650+0000","last_unstale":"2024-08-17T02:46:16.786226+0000","last_undegraded":"2024-08-17T02:46:16.786226+0000","last_fullsized":"2024-08-17T02:46:16.786226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:51:11.224931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.786106+0000","last_change":"2024-08-17T02:46:15.784816+0000","last_active":"2024-08-17T02:46:15.786106+0000","last_peered":"2024-08-17T02:46:15.786106+0000","last_clean":"2024-08-17T02:46:15.786106+0000","last_became_active":"2024-08-17T02:46:15.784489+0000","last_became_peered":"2024-08-17T02:46:15.784489+0000","last_unstale":"2024-08-17T02:46:15.786106+0000","last_undegraded":"2024-08-17T02:46:15.786106+0000","last_fullsized":"2024-08-17T02:46:15.786106+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:37:24.295411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.778727+0000","last_change":"2024-08-17T02:46:16.778727+0000","last_active":"2024-08-17T02:46:16.778727+0000","last_peered":"2024-08-17T02:46:16.778727+0000","last_clean":"2024-08-17T02:46:16.778727+0000","last_became_active":"2024-08-17T02:46:16.778387+0000","last_became_peered":"2024-08-17T02:46:16.778387+0000","last_unstale":"2024-08-17T02:46:16.778727+0000","last_undegraded":"2024-08-17T02:46:16.778727+0000","last_fullsized":"2024-08-17T02:46:16.778727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:13:16.818070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.802526+0000","last_change":"2024-08-17T02:46:15.774187+0000","last_active":"2024-08-17T02:46:16.802526+0000","last_peered":"2024-08-17T02:46:16.802526+0000","last_clean":"2024-08-17T02:46:16.802526+0000","last_became_active":"2024-08-17T02:46:15.773919+0000","last_became_peered":"2024-08-17T02:46:15.773919+0000","last_unstale":"2024-08-17T02:46:16.802526+0000","last_undegraded":"2024-08-17T02:46:16.802526+0000","last_fullsized":"2024-08-17T02:46:16.802526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:37:24.295411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.777955+0000","last_change":"2024-08-17T02:46:16.777955+0000","last_active":"2024-08-17T02:46:16.777955+0000","last_peered":"2024-08-17T02:46:16.777955+0000","last_clean":"2024-08-17T02:46:16.777955+0000","last_became_active":"2024-08-17T02:46:16.777500+0000","last_became_peered":"2024-08-17T02:46:16.777500+0000","last_unstale":"2024-08-17T02:46:16.777955+0000","last_undegraded":"2024-08-17T02:46:16.777955+0000","last_fullsized":"2024-08-17T02:46:16.777955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:02:07.541310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.766201+0000","last_change":"2024-08-17T02:46:15.775398+0000","last_active":"2024-08-17T02:46:16.766201+0000","last_peered":"2024-08-17T02:46:16.766201+0000","last_clean":"2024-08-17T02:46:16.766201+0000","last_became_active":"2024-08-17T02:46:15.775106+0000","last_became_peered":"2024-08-17T02:46:15.775106+0000","last_unstale":"2024-08-17T02:46:16.766201+0000","last_undegraded":"2024-08-17T02:46:16.766201+0000","last_fullsized":"2024-08-17T02:46:16.766201+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:51:10.218325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.787186+0000","last_change":"2024-08-17T02:46:16.787186+0000","last_active":"2024-08-17T02:46:16.787186+0000","last_peered":"2024-08-17T02:46:16.787186+0000","last_clean":"2024-08-17T02:46:16.787186+0000","last_became_active":"2024-08-17T02:46:16.786893+0000","last_became_peered":"2024-08-17T02:46:16.786893+0000","last_unstale":"2024-08-17T02:46:16.787186+0000","last_undegraded":"2024-08-17T02:46:16.787186+0000","last_fullsized":"2024-08-17T02:46:16.787186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:39:22.034306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.797571+0000","last_change":"2024-08-17T02:46:15.779121+0000","last_active":"2024-08-17T02:46:16.797571+0000","last_peered":"2024-08-17T02:46:16.797571+0000","last_clean":"2024-08-17T02:46:16.797571+0000","last_became_active":"2024-08-17T02:46:15.778318+0000","last_became_peered":"2024-08-17T02:46:15.778318+0000","last_unstale":"2024-08-17T02:46:16.797571+0000","last_undegraded":"2024-08-17T02:46:16.797571+0000","last_fullsized":"2024-08-17T02:46:16.797571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:38:54.397608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.806933+0000","last_change":"2024-08-17T02:46:16.800899+0000","last_active":"2024-08-17T02:46:16.806933+0000","last_peered":"2024-08-17T02:46:16.806933+0000","last_clean":"2024-08-17T02:46:16.806933+0000","last_became_active":"2024-08-17T02:46:16.798624+0000","last_became_peered":"2024-08-17T02:46:16.798624+0000","last_unstale":"2024-08-17T02:46:16.806933+0000","last_undegraded":"2024-08-17T02:46:16.806933+0000","last_fullsized":"2024-08-17T02:46:16.806933+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:27:15.747236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.786322+0000","last_change":"2024-08-17T02:46:15.783352+0000","last_active":"2024-08-17T02:46:15.786322+0000","last_peered":"2024-08-17T02:46:15.786322+0000","last_clean":"2024-08-17T02:46:15.786322+0000","last_became_active":"2024-08-17T02:46:15.783075+0000","last_became_peered":"2024-08-17T02:46:15.783075+0000","last_unstale":"2024-08-17T02:46:15.786322+0000","last_undegraded":"2024-08-17T02:46:15.786322+0000","last_fullsized":"2024-08-17T02:46:15.786322+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:32:00.682781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.797310+0000","last_change":"2024-08-17T02:46:16.794091+0000","last_active":"2024-08-17T02:46:16.797310+0000","last_peered":"2024-08-17T02:46:16.797310+0000","last_clean":"2024-08-17T02:46:16.797310+0000","last_became_active":"2024-08-17T02:46:16.793580+0000","last_became_peered":"2024-08-17T02:46:16.793580+0000","last_unstale":"2024-08-17T02:46:16.797310+0000","last_undegraded":"2024-08-17T02:46:16.797310+0000","last_fullsized":"2024-08-17T02:46:16.797310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:32:58.044719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.803238+0000","last_change":"2024-08-17T02:46:15.781177+0000","last_active":"2024-08-17T02:46:16.803238+0000","last_peered":"2024-08-17T02:46:16.803238+0000","last_clean":"2024-08-17T02:46:16.803238+0000","last_became_active":"2024-08-17T02:46:15.780921+0000","last_became_peered":"2024-08-17T02:46:15.780921+0000","last_unstale":"2024-08-17T02:46:16.803238+0000","last_undegraded":"2024-08-17T02:46:16.803238+0000","last_fullsized":"2024-08-17T02:46:16.803238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:31:49.347928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.782404+0000","last_change":"2024-08-17T02:46:16.782404+0000","last_active":"2024-08-17T02:46:16.782404+0000","last_peered":"2024-08-17T02:46:16.782404+0000","last_clean":"2024-08-17T02:46:16.782404+0000","last_became_active":"2024-08-17T02:46:16.781487+0000","last_became_peered":"2024-08-17T02:46:16.781487+0000","last_unstale":"2024-08-17T02:46:16.782404+0000","last_undegraded":"2024-08-17T02:46:16.782404+0000","last_fullsized":"2024-08-17T02:46:16.782404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:12:01.932427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.783484+0000","last_change":"2024-08-17T02:46:15.776778+0000","last_active":"2024-08-17T02:46:15.783484+0000","last_peered":"2024-08-17T02:46:15.783484+0000","last_clean":"2024-08-17T02:46:15.783484+0000","last_became_active":"2024-08-17T02:46:15.776448+0000","last_became_peered":"2024-08-17T02:46:15.776448+0000","last_unstale":"2024-08-17T02:46:15.783484+0000","last_undegraded":"2024-08-17T02:46:15.783484+0000","last_fullsized":"2024-08-17T02:46:15.783484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:55:59.916719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.791248+0000","last_change":"2024-08-17T02:46:16.787985+0000","last_active":"2024-08-17T02:46:16.791248+0000","last_peered":"2024-08-17T02:46:16.791248+0000","last_clean":"2024-08-17T02:46:16.791248+0000","last_became_active":"2024-08-17T02:46:16.787668+0000","last_became_peered":"2024-08-17T02:46:16.787668+0000","last_unstale":"2024-08-17T02:46:16.791248+0000","last_undegraded":"2024-08-17T02:46:16.791248+0000","last_fullsized":"2024-08-17T02:46:16.791248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:37:25.302017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.784183+0000","last_change":"2024-08-17T02:46:15.777081+0000","last_active":"2024-08-17T02:46:15.784183+0000","last_peered":"2024-08-17T02:46:15.784183+0000","last_clean":"2024-08-17T02:46:15.784183+0000","last_became_active":"2024-08-17T02:46:15.776804+0000","last_became_peered":"2024-08-17T02:46:15.776804+0000","last_unstale":"2024-08-17T02:46:15.784183+0000","last_undegraded":"2024-08-17T02:46:15.784183+0000","last_fullsized":"2024-08-17T02:46:15.784183+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:43:13.086847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.789235+0000","last_change":"2024-08-17T02:46:16.789235+0000","last_active":"2024-08-17T02:46:16.789235+0000","last_peered":"2024-08-17T02:46:16.789235+0000","last_clean":"2024-08-17T02:46:16.789235+0000","last_became_active":"2024-08-17T02:46:16.782691+0000","last_became_peered":"2024-08-17T02:46:16.782691+0000","last_unstale":"2024-08-17T02:46:16.789235+0000","last_undegraded":"2024-08-17T02:46:16.789235+0000","last_fullsized":"2024-08-17T02:46:16.789235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:32:01.689387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.784252+0000","last_change":"2024-08-17T02:46:15.780992+0000","last_active":"2024-08-17T02:46:15.784252+0000","last_peered":"2024-08-17T02:46:15.784252+0000","last_clean":"2024-08-17T02:46:15.784252+0000","last_became_active":"2024-08-17T02:46:15.780752+0000","last_became_peered":"2024-08-17T02:46:15.780752+0000","last_unstale":"2024-08-17T02:46:15.784252+0000","last_undegraded":"2024-08-17T02:46:15.784252+0000","last_fullsized":"2024-08-17T02:46:15.784252+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:47:08.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.788115+0000","last_change":"2024-08-17T02:46:16.783355+0000","last_active":"2024-08-17T02:46:16.788115+0000","last_peered":"2024-08-17T02:46:16.788115+0000","last_clean":"2024-08-17T02:46:16.788115+0000","last_became_active":"2024-08-17T02:46:16.782953+0000","last_became_peered":"2024-08-17T02:46:16.782953+0000","last_unstale":"2024-08-17T02:46:16.788115+0000","last_undegraded":"2024-08-17T02:46:16.788115+0000","last_fullsized":"2024-08-17T02:46:16.788115+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:34:26.609150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.786791+0000","last_change":"2024-08-17T02:46:15.782313+0000","last_active":"2024-08-17T02:46:15.786791+0000","last_peered":"2024-08-17T02:46:15.786791+0000","last_clean":"2024-08-17T02:46:15.786791+0000","last_became_active":"2024-08-17T02:46:15.782068+0000","last_became_peered":"2024-08-17T02:46:15.782068+0000","last_unstale":"2024-08-17T02:46:15.786791+0000","last_undegraded":"2024-08-17T02:46:15.786791+0000","last_fullsized":"2024-08-17T02:46:15.786791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:03.733478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.791773+0000","last_change":"2024-08-17T02:46:16.790570+0000","last_active":"2024-08-17T02:46:16.791773+0000","last_peered":"2024-08-17T02:46:16.791773+0000","last_clean":"2024-08-17T02:46:16.791773+0000","last_became_active":"2024-08-17T02:46:16.790301+0000","last_became_peered":"2024-08-17T02:46:16.790301+0000","last_unstale":"2024-08-17T02:46:16.791773+0000","last_undegraded":"2024-08-17T02:46:16.791773+0000","last_fullsized":"2024-08-17T02:46:16.791773+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:24:25.550110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.766592+0000","last_change":"2024-08-17T02:46:15.779038+0000","last_active":"2024-08-17T02:46:16.766592+0000","last_peered":"2024-08-17T02:46:16.766592+0000","last_clean":"2024-08-17T02:46:16.766592+0000","last_became_active":"2024-08-17T02:46:15.775267+0000","last_became_peered":"2024-08-17T02:46:15.775267+0000","last_unstale":"2024-08-17T02:46:16.766592+0000","last_undegraded":"2024-08-17T02:46:16.766592+0000","last_fullsized":"2024-08-17T02:46:16.766592+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:43:34.809998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.783698+0000","last_change":"2024-08-17T02:46:16.783698+0000","last_active":"2024-08-17T02:46:16.783698+0000","last_peered":"2024-08-17T02:46:16.783698+0000","last_clean":"2024-08-17T02:46:16.783698+0000","last_became_active":"2024-08-17T02:46:16.781902+0000","last_became_peered":"2024-08-17T02:46:16.781902+0000","last_unstale":"2024-08-17T02:46:16.783698+0000","last_undegraded":"2024-08-17T02:46:16.783698+0000","last_fullsized":"2024-08-17T02:46:16.783698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:24:54.295510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.776850+0000","last_change":"2024-08-17T02:46:15.780976+0000","last_active":"2024-08-17T02:46:16.776850+0000","last_peered":"2024-08-17T02:46:16.776850+0000","last_clean":"2024-08-17T02:46:16.776850+0000","last_became_active":"2024-08-17T02:46:15.780616+0000","last_became_peered":"2024-08-17T02:46:15.780616+0000","last_unstale":"2024-08-17T02:46:16.776850+0000","last_undegraded":"2024-08-17T02:46:16.776850+0000","last_fullsized":"2024-08-17T02:46:16.776850+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:37:24.295411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.787791+0000","last_change":"2024-08-17T02:46:16.787791+0000","last_active":"2024-08-17T02:46:16.787791+0000","last_peered":"2024-08-17T02:46:16.787791+0000","last_clean":"2024-08-17T02:46:16.787791+0000","last_became_active":"2024-08-17T02:46:16.787571+0000","last_became_peered":"2024-08-17T02:46:16.787571+0000","last_unstale":"2024-08-17T02:46:16.787791+0000","last_undegraded":"2024-08-17T02:46:16.787791+0000","last_fullsized":"2024-08-17T02:46:16.787791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:19:59.271493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.787137+0000","last_change":"2024-08-17T02:46:15.785899+0000","last_active":"2024-08-17T02:46:15.787137+0000","last_peered":"2024-08-17T02:46:15.787137+0000","last_clean":"2024-08-17T02:46:15.787137+0000","last_became_active":"2024-08-17T02:46:15.785632+0000","last_became_peered":"2024-08-17T02:46:15.785632+0000","last_unstale":"2024-08-17T02:46:15.787137+0000","last_undegraded":"2024-08-17T02:46:15.787137+0000","last_fullsized":"2024-08-17T02:46:15.787137+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:44:21.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.793162+0000","last_change":"2024-08-17T02:45:16.001117+0000","last_active":"2024-08-17T02:46:16.793162+0000","last_peered":"2024-08-17T02:46:16.793162+0000","last_clean":"2024-08-17T02:46:16.793162+0000","last_became_active":"2024-08-17T02:45:16.000788+0000","last_became_peered":"2024-08-17T02:45:16.000788+0000","last_unstale":"2024-08-17T02:46:16.793162+0000","last_undegraded":"2024-08-17T02:46:16.793162+0000","last_fullsized":"2024-08-17T02:46:16.793162+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_clean_scrub_stamp":"2024-08-17T02:45:14.986707+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:27:37.295374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":364544,"data_stored":137418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.8780000000000001,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55700000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.85899999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":4.6870000000000003,"5min":4.6870000000000003,"15min":4.6870000000000003},"last":0.878}]},{"osd":3,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.66800000000000004}]},{"osd":4,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80000000000000004}]},{"osd":5,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.91900000000000004}]},{"osd":6,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.753},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.70899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297724,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074116,"statfs":{"total":96636764160,"available":96331894784,"internally_reserved":0,"allocated":372736,"data_stored":138948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.47699999999999998}]},{"osd":1,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.52000000000000002}]},{"osd":2,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70999999999999996}]},{"osd":3,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":4.0570000000000004,"5min":4.0570000000000004,"15min":4.0570000000000004},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.86399999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.80300000000000005}]},{"osd":7,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.748},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":1.405}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073544,"statfs":{"total":96636764160,"available":96331309056,"internally_reserved":0,"allocated":958464,"data_stored":728334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.80100000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.71199999999999997}]},{"osd":2,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72399999999999998}]},{"osd":3,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.84899999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.496},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55300000000000005}]},{"osd":6,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59699999999999998}]},{"osd":7,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.73999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297736,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074104,"statfs":{"total":96636764160,"available":96331882496,"internally_reserved":0,"allocated":389120,"data_stored":145258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.78000000000000003}]},{"osd":1,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.74399999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.623},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.76400000000000001}]},{"osd":5,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84199999999999997}]},{"osd":6,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.81200000000000006}]},{"osd":7,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.72499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":376832,"data_stored":138160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.311},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.552,"5min":3.552,"15min":3.552},"last":0.41799999999999998}]},{"osd":1,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.44}]},{"osd":2,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.8149999999999999,"5min":3.8149999999999999,"15min":3.8149999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":0.48699999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":3.9009999999999998,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":4.3360000000000003,"5min":4.3360000000000003,"15min":4.3360000000000003},"last":0.85199999999999998}]},{"osd":5,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":3.371,"5min":3.371,"15min":3.371},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.93400000000000005}]},{"osd":6,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":4.4160000000000004,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":0.91200000000000003}]},{"osd":7,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":4.3799999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.89900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301880,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069960,"statfs":{"total":96636764160,"available":96327639040,"internally_reserved":0,"allocated":364544,"data_stored":137430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.373},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.48199999999999998}]},{"osd":3,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":7.5960000000000001,"5min":7.5960000000000001,"15min":7.5960000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":6.0499999999999998,"5min":6.0499999999999998,"15min":6.0499999999999998},"last":0.53600000000000003}]},{"osd":4,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":5.7009999999999996,"5min":5.7009999999999996,"15min":5.7009999999999996},"last":0.73699999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":7.5220000000000002,"5min":7.5220000000000002,"15min":7.5220000000000002},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":0.60599999999999998}]},{"osd":6,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":5.8780000000000001,"5min":5.8780000000000001,"15min":5.8780000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":7.5570000000000004,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.63800000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":5.9119999999999999,"5min":5.9119999999999999,"15min":5.9119999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":7.4800000000000004,"5min":7.4800000000000004,"15min":7.4800000000000004},"last":0.71199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":970752,"data_stored":734122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.34599999999999997}]},{"osd":2,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.45400000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.495},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.9399999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.41499999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.71499999999999997}]},{"osd":6,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.5059999999999998,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":0.69499999999999995}]},{"osd":7,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.75600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301880,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069960,"statfs":{"total":96636764160,"available":96327639040,"internally_reserved":0,"allocated":364544,"data_stored":137486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.39700000000000002}]},{"osd":2,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.47999999999999998}]},{"osd":3,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.53700000000000003}]},{"osd":4,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.747},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.82899999999999996}]},{"osd":5,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.85599999999999998}]},{"osd":7,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.874}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:46:21.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:46:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.191+0000 7f8011200640 1 Processor -- start 2024-08-17T02:46:21.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.192+0000 7f8011200640 1 -- start start 2024-08-17T02:46:21.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.193+0000 7f8011200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f800c0b0270 0x7f800c1563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:21.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.193+0000 7f8011200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800c157690 con 0x7f800c158b30 2024-08-17T02:46:21.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.193+0000 7f8011200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800c156a80 con 0x7f800c155900 2024-08-17T02:46:21.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.193+0000 7f8011200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800c156d60 con 0x7f800c0b0270 2024-08-17T02:46:21.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.193+0000 7f8003e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f800c0b0270 0x7f800c1563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.193+0000 7f8003e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f800c0b0270 0x7f800c1563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43112/0 (socket says 172.21.15.16:43112) 2024-08-17T02:46:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.193+0000 7f8003e00640 1 -- 172.21.15.16:0/2766165969 learned_addr learned my addr 172.21.15.16:0/2766165969 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.193+0000 7f800be00640 1 --1- 172.21.15.16:0/2766165969 >> v1:172.21.15.16:6789/0 conn(0x7f800c158b30 0x7f800c0ae6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57298/0 (socket says 172.21.15.16:57298) 2024-08-17T02:46:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.194+0000 7f8003e00640 1 -- 172.21.15.16:0/2766165969 >> v1:172.21.15.142:6789/0 conn(0x7f800c155900 legacy=0x7f800c155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.194+0000 7f8003e00640 1 -- 172.21.15.16:0/2766165969 >> v1:172.21.15.16:6789/0 conn(0x7f800c158b30 legacy=0x7f800c0ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.194+0000 7f8003e00640 1 -- 172.21.15.16:0/2766165969 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c157040 con 0x7f800c0b0270 2024-08-17T02:46:21.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.194+0000 7f8003e00640 1 --2- 172.21.15.16:0/2766165969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800c0b0270 0x7f800c1563e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009910 tx=0x7f7ff40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc3c964aaa90182 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:21.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.195+0000 7f800aa00640 1 -- 172.21.15.16:0/2766165969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4032440 con 0x7f800c0b0270 2024-08-17T02:46:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.195+0000 7f800aa00640 1 -- 172.21.15.16:0/2766165969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff40325a0 con 0x7f800c0b0270 2024-08-17T02:46:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.195+0000 7f800aa00640 1 -- 172.21.15.16:0/2766165969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4032980 con 0x7f800c0b0270 2024-08-17T02:46:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.195+0000 7f8011200640 1 -- 172.21.15.16:0/2766165969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800c0b0270 msgr2=0x7f800c1563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.195+0000 7f8011200640 1 --2- 172.21.15.16:0/2766165969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800c0b0270 0x7f800c1563e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009910 tx=0x7f7ff40091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.195+0000 7f8011200640 1 -- 172.21.15.16:0/2766165969 shutdown_connections 2024-08-17T02:46:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.195+0000 7f8011200640 1 --2- 172.21.15.16:0/2766165969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800c0b0270 0x7f800c1563e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.195+0000 7f8011200640 1 -- 172.21.15.16:0/2766165969 >> 172.21.15.16:0/2766165969 conn(0x7f800c067750 msgr2=0x7f800c0cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.195+0000 7f8011200640 1 -- 172.21.15.16:0/2766165969 shutdown_connections 2024-08-17T02:46:21.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.195+0000 7f8011200640 1 -- 172.21.15.16:0/2766165969 wait complete. 2024-08-17T02:46:21.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.197+0000 7f8011200640 1 Processor -- start 2024-08-17T02:46:21.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.197+0000 7f8011200640 1 -- start start 2024-08-17T02:46:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.198+0000 7f8011200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f800c0b0270 0x7f800c0a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.198+0000 7f8011200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f800c155900 0x7f800c1408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.198+0000 7f8011200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800c158b30 0x7f800c0c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.198+0000 7f8011200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0ab080 con 0x7f800c158b30 2024-08-17T02:46:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.198+0000 7f8011200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0a6430 con 0x7f800c0b0270 2024-08-17T02:46:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.198+0000 7f8011200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0a9c00 con 0x7f800c155900 2024-08-17T02:46:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.198+0000 7f800be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800c158b30 0x7f800c0c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.198+0000 7f800be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800c158b30 0x7f800c0c9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43118/0 (socket says 172.21.15.16:43118) 2024-08-17T02:46:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.198+0000 7f800be00640 1 -- 172.21.15.16:0/4213242747 learned_addr learned my addr 172.21.15.16:0/4213242747 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.198+0000 7f800b400640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f800c0b0270 0x7f800c0a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.198+0000 7f8003e00640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f800c155900 0x7f800c1408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:21.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.199+0000 7f800be00640 1 -- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f800c155900 msgr2=0x7f800c1408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.199+0000 7f800be00640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f800c155900 0x7f800c1408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.199+0000 7f800be00640 1 -- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f800c0b0270 msgr2=0x7f800c0a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.199+0000 7f800be00640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f800c0b0270 0x7f800c0a96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.199+0000 7f800be00640 1 -- 172.21.15.16:0/4213242747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff4009450 con 0x7f800c158b30 2024-08-17T02:46:21.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.199+0000 7f800be00640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800c158b30 0x7f800c0c9a70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f800400c840 tx=0x7f80040015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.199+0000 7f8002a00640 1 -- 172.21.15.16:0/4213242747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800400ce30 con 0x7f800c158b30 2024-08-17T02:46:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.199+0000 7f8002a00640 1 -- 172.21.15.16:0/4213242747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8004005d60 con 0x7f800c158b30 2024-08-17T02:46:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.199+0000 7f8002a00640 1 -- 172.21.15.16:0/4213242747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800400a810 con 0x7f800c158b30 2024-08-17T02:46:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.199+0000 7f8011200640 1 -- 172.21.15.16:0/4213242747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c0a5910 con 0x7f800c158b30 2024-08-17T02:46:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.199+0000 7f8011200640 1 -- 172.21.15.16:0/4213242747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c0a6150 con 0x7f800c158b30 2024-08-17T02:46:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.200+0000 7f8011200640 1 -- 172.21.15.16:0/4213242747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c155d40 con 0x7f800c158b30 2024-08-17T02:46:21.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.201+0000 7f8002a00640 1 -- 172.21.15.16:0/4213242747 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f800400ac40 con 0x7f800c158b30 2024-08-17T02:46:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.202+0000 7f8002a00640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7fe8065150 0x7f7fe8067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.202+0000 7f8002a00640 1 -- 172.21.15.16:0/4213242747 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8004082020 con 0x7f800c158b30 2024-08-17T02:46:21.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.202+0000 7f800b400640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7fe8065150 0x7f7fe8067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:21.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.203+0000 7f800b400640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7fe8065150 0x7f7fe8067600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc007fd0 tx=0x7f7ffc007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:21.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.206+0000 7f8002a00640 1 -- 172.21.15.16:0/4213242747 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f800c155d40 con 0x7f800c158b30 2024-08-17T02:46:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.384+0000 7f8011200640 1 -- 172.21.15.16:0/4213242747 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f800c155d40 con 0x7f7fe8065150 2024-08-17T02:46:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.392+0000 7f8002a00640 1 -- 172.21.15.16:0/4213242747 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288953 (secure 0 0 0) 0x7f800c155d40 con 0x7f7fe8065150 2024-08-17T02:46:21.395 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:46:21.397 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-17T02:46:21.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.396+0000 7f8011200640 1 -- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7fe8065150 msgr2=0x7f7fe8067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.396+0000 7f8011200640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7fe8065150 0x7f7fe8067600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc007fd0 tx=0x7f7ffc007e30 comp rx=0 tx=0).stop 2024-08-17T02:46:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.396+0000 7f8011200640 1 -- 172.21.15.16:0/4213242747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800c158b30 msgr2=0x7f800c0c9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.396+0000 7f8011200640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800c158b30 0x7f800c0c9a70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f800400c840 tx=0x7f80040015f0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.396+0000 7f8011200640 1 -- 172.21.15.16:0/4213242747 shutdown_connections 2024-08-17T02:46:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.396+0000 7f8011200640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7fe8065150 0x7f7fe8067600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.396+0000 7f8011200640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800c158b30 0x7f800c0c9a70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.396+0000 7f8011200640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f800c155900 0x7f800c1408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.396+0000 7f8011200640 1 --2- 172.21.15.16:0/4213242747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f800c0b0270 0x7f800c0a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.396+0000 7f8011200640 1 -- 172.21.15.16:0/4213242747 >> 172.21.15.16:0/4213242747 conn(0x7f800c067750 msgr2=0x7f800c068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.396+0000 7f8011200640 1 -- 172.21.15.16:0/4213242747 shutdown_connections 2024-08-17T02:46:21.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.396+0000 7f8011200640 1 -- 172.21.15.16:0/4213242747 wait complete. 2024-08-17T02:46:21.411 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-08-17T02:46:20.250198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2399600,"kb_used_data":4064,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752575120,"statfs":{"total":773094113280,"available":770636922880,"internally_reserved":0,"allocated":4161536,"data_stored":2297156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.499790"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.798064+0000","last_change":"2024-08-17T02:46:16.780710+0000","last_active":"2024-08-17T02:46:16.798064+0000","last_peered":"2024-08-17T02:46:16.798064+0000","last_clean":"2024-08-17T02:46:16.798064+0000","last_became_active":"2024-08-17T02:46:16.780129+0000","last_became_peered":"2024-08-17T02:46:16.780129+0000","last_unstale":"2024-08-17T02:46:16.798064+0000","last_undegraded":"2024-08-17T02:46:16.798064+0000","last_fullsized":"2024-08-17T02:46:16.798064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:06:28.508134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.788411+0000","last_change":"2024-08-17T02:46:16.788411+0000","last_active":"2024-08-17T02:46:16.788411+0000","last_peered":"2024-08-17T02:46:16.788411+0000","last_clean":"2024-08-17T02:46:16.788411+0000","last_became_active":"2024-08-17T02:46:16.786270+0000","last_became_peered":"2024-08-17T02:46:16.786270+0000","last_unstale":"2024-08-17T02:46:16.788411+0000","last_undegraded":"2024-08-17T02:46:16.788411+0000","last_fullsized":"2024-08-17T02:46:16.788411+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:32:01.689387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.807045+0000","last_change":"2024-08-17T02:46:16.803017+0000","last_active":"2024-08-17T02:46:16.807045+0000","last_peered":"2024-08-17T02:46:16.807045+0000","last_clean":"2024-08-17T02:46:16.807045+0000","last_became_active":"2024-08-17T02:46:16.802772+0000","last_became_peered":"2024-08-17T02:46:16.802772+0000","last_unstale":"2024-08-17T02:46:16.807045+0000","last_undegraded":"2024-08-17T02:46:16.807045+0000","last_fullsized":"2024-08-17T02:46:16.807045+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:31:50.354534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.790489+0000","last_change":"2024-08-17T02:46:16.786268+0000","last_active":"2024-08-17T02:46:16.790489+0000","last_peered":"2024-08-17T02:46:16.790489+0000","last_clean":"2024-08-17T02:46:16.790489+0000","last_became_active":"2024-08-17T02:46:16.786015+0000","last_became_peered":"2024-08-17T02:46:16.786015+0000","last_unstale":"2024-08-17T02:46:16.790489+0000","last_undegraded":"2024-08-17T02:46:16.790489+0000","last_fullsized":"2024-08-17T02:46:16.790489+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:43:02.974223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.772747+0000","last_change":"2024-08-17T02:46:16.772747+0000","last_active":"2024-08-17T02:46:16.772747+0000","last_peered":"2024-08-17T02:46:16.772747+0000","last_clean":"2024-08-17T02:46:16.772747+0000","last_became_active":"2024-08-17T02:46:16.772510+0000","last_became_peered":"2024-08-17T02:46:16.772510+0000","last_unstale":"2024-08-17T02:46:16.772747+0000","last_undegraded":"2024-08-17T02:46:16.772747+0000","last_fullsized":"2024-08-17T02:46:16.772747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:13:45.059552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.789758+0000","last_change":"2024-08-17T02:46:16.785700+0000","last_active":"2024-08-17T02:46:16.789758+0000","last_peered":"2024-08-17T02:46:16.789758+0000","last_clean":"2024-08-17T02:46:16.789758+0000","last_became_active":"2024-08-17T02:46:16.785413+0000","last_became_peered":"2024-08-17T02:46:16.785413+0000","last_unstale":"2024-08-17T02:46:16.789758+0000","last_undegraded":"2024-08-17T02:46:16.789758+0000","last_fullsized":"2024-08-17T02:46:16.789758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:47:09.835864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.808530+0000","last_change":"2024-08-17T02:46:16.802806+0000","last_active":"2024-08-17T02:46:16.808530+0000","last_peered":"2024-08-17T02:46:16.808530+0000","last_clean":"2024-08-17T02:46:16.808530+0000","last_became_active":"2024-08-17T02:46:16.802495+0000","last_became_peered":"2024-08-17T02:46:16.802495+0000","last_unstale":"2024-08-17T02:46:16.808530+0000","last_undegraded":"2024-08-17T02:46:16.808530+0000","last_fullsized":"2024-08-17T02:46:16.808530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:43.890584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.789538+0000","last_change":"2024-08-17T02:46:16.779804+0000","last_active":"2024-08-17T02:46:16.789538+0000","last_peered":"2024-08-17T02:46:16.789538+0000","last_clean":"2024-08-17T02:46:16.789538+0000","last_became_active":"2024-08-17T02:46:16.779553+0000","last_became_peered":"2024-08-17T02:46:16.779553+0000","last_unstale":"2024-08-17T02:46:16.789538+0000","last_undegraded":"2024-08-17T02:46:16.789538+0000","last_fullsized":"2024-08-17T02:46:16.789538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:44:22.048521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.796842+0000","last_change":"2024-08-17T02:46:16.790616+0000","last_active":"2024-08-17T02:46:16.796842+0000","last_peered":"2024-08-17T02:46:16.796842+0000","last_clean":"2024-08-17T02:46:16.796842+0000","last_became_active":"2024-08-17T02:46:16.790362+0000","last_became_peered":"2024-08-17T02:46:16.790362+0000","last_unstale":"2024-08-17T02:46:16.796842+0000","last_undegraded":"2024-08-17T02:46:16.796842+0000","last_fullsized":"2024-08-17T02:46:16.796842+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:51:11.224931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.797531+0000","last_change":"2024-08-17T02:46:16.792038+0000","last_active":"2024-08-17T02:46:16.797531+0000","last_peered":"2024-08-17T02:46:16.797531+0000","last_clean":"2024-08-17T02:46:16.797531+0000","last_became_active":"2024-08-17T02:46:16.791769+0000","last_became_peered":"2024-08-17T02:46:16.791769+0000","last_unstale":"2024-08-17T02:46:16.797531+0000","last_undegraded":"2024-08-17T02:46:16.797531+0000","last_fullsized":"2024-08-17T02:46:16.797531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:44:38.564320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.786660+0000","last_change":"2024-08-17T02:46:16.786660+0000","last_active":"2024-08-17T02:46:16.786660+0000","last_peered":"2024-08-17T02:46:16.786660+0000","last_clean":"2024-08-17T02:46:16.786660+0000","last_became_active":"2024-08-17T02:46:16.786057+0000","last_became_peered":"2024-08-17T02:46:16.786057+0000","last_unstale":"2024-08-17T02:46:16.786660+0000","last_undegraded":"2024-08-17T02:46:16.786660+0000","last_fullsized":"2024-08-17T02:46:16.786660+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:48:48.175556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.799505+0000","last_change":"2024-08-17T02:46:16.790194+0000","last_active":"2024-08-17T02:46:16.799505+0000","last_peered":"2024-08-17T02:46:16.799505+0000","last_clean":"2024-08-17T02:46:16.799505+0000","last_became_active":"2024-08-17T02:46:16.789942+0000","last_became_peered":"2024-08-17T02:46:16.789942+0000","last_unstale":"2024-08-17T02:46:16.799505+0000","last_undegraded":"2024-08-17T02:46:16.799505+0000","last_fullsized":"2024-08-17T02:46:16.799505+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:43:14.093453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.781804+0000","last_change":"2024-08-17T02:46:16.781804+0000","last_active":"2024-08-17T02:46:16.781804+0000","last_peered":"2024-08-17T02:46:16.781804+0000","last_clean":"2024-08-17T02:46:16.781804+0000","last_became_active":"2024-08-17T02:46:16.781527+0000","last_became_peered":"2024-08-17T02:46:16.781527+0000","last_unstale":"2024-08-17T02:46:16.781804+0000","last_undegraded":"2024-08-17T02:46:16.781804+0000","last_fullsized":"2024-08-17T02:46:16.781804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:02:07.541310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.785745+0000","last_change":"2024-08-17T02:46:16.780339+0000","last_active":"2024-08-17T02:46:16.785745+0000","last_peered":"2024-08-17T02:46:16.785745+0000","last_clean":"2024-08-17T02:46:16.785745+0000","last_became_active":"2024-08-17T02:46:16.779588+0000","last_became_peered":"2024-08-17T02:46:16.779588+0000","last_unstale":"2024-08-17T02:46:16.785745+0000","last_undegraded":"2024-08-17T02:46:16.785745+0000","last_fullsized":"2024-08-17T02:46:16.785745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:19:59.271493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.780568+0000","last_change":"2024-08-17T02:46:16.780568+0000","last_active":"2024-08-17T02:46:16.780568+0000","last_peered":"2024-08-17T02:46:16.780568+0000","last_clean":"2024-08-17T02:46:16.780568+0000","last_became_active":"2024-08-17T02:46:16.780268+0000","last_became_peered":"2024-08-17T02:46:16.780268+0000","last_unstale":"2024-08-17T02:46:16.780568+0000","last_undegraded":"2024-08-17T02:46:16.780568+0000","last_fullsized":"2024-08-17T02:46:16.780568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T14:13:45.059552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.783449+0000","last_change":"2024-08-17T02:46:16.783449+0000","last_active":"2024-08-17T02:46:16.783449+0000","last_peered":"2024-08-17T02:46:16.783449+0000","last_clean":"2024-08-17T02:46:16.783449+0000","last_became_active":"2024-08-17T02:46:16.782513+0000","last_became_peered":"2024-08-17T02:46:16.782513+0000","last_unstale":"2024-08-17T02:46:16.783449+0000","last_undegraded":"2024-08-17T02:46:16.783449+0000","last_fullsized":"2024-08-17T02:46:16.783449+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:59:23.376039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.783117+0000","last_change":"2024-08-17T02:46:16.783117+0000","last_active":"2024-08-17T02:46:16.783117+0000","last_peered":"2024-08-17T02:46:16.783117+0000","last_clean":"2024-08-17T02:46:16.783117+0000","last_became_active":"2024-08-17T02:46:16.781944+0000","last_became_peered":"2024-08-17T02:46:16.781944+0000","last_unstale":"2024-08-17T02:46:16.783117+0000","last_undegraded":"2024-08-17T02:46:16.783117+0000","last_fullsized":"2024-08-17T02:46:16.783117+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:21:47.106619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.786286+0000","last_change":"2024-08-17T02:46:16.783330+0000","last_active":"2024-08-17T02:46:16.786286+0000","last_peered":"2024-08-17T02:46:16.786286+0000","last_clean":"2024-08-17T02:46:16.786286+0000","last_became_active":"2024-08-17T02:46:16.782741+0000","last_became_peered":"2024-08-17T02:46:16.782741+0000","last_unstale":"2024-08-17T02:46:16.786286+0000","last_undegraded":"2024-08-17T02:46:16.786286+0000","last_fullsized":"2024-08-17T02:46:16.786286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:03:00.932909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.792670+0000","last_change":"2024-08-17T02:46:16.791359+0000","last_active":"2024-08-17T02:46:16.792670+0000","last_peered":"2024-08-17T02:46:16.792670+0000","last_clean":"2024-08-17T02:46:16.792670+0000","last_became_active":"2024-08-17T02:46:16.791083+0000","last_became_peered":"2024-08-17T02:46:16.791083+0000","last_unstale":"2024-08-17T02:46:16.792670+0000","last_undegraded":"2024-08-17T02:46:16.792670+0000","last_fullsized":"2024-08-17T02:46:16.792670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:43:35.816603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.778355+0000","last_change":"2024-08-17T02:46:16.778355+0000","last_active":"2024-08-17T02:46:16.778355+0000","last_peered":"2024-08-17T02:46:16.778355+0000","last_clean":"2024-08-17T02:46:16.778355+0000","last_became_active":"2024-08-17T02:46:16.778072+0000","last_became_peered":"2024-08-17T02:46:16.778072+0000","last_unstale":"2024-08-17T02:46:16.778355+0000","last_undegraded":"2024-08-17T02:46:16.778355+0000","last_fullsized":"2024-08-17T02:46:16.778355+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:23:25.110064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.775559+0000","last_change":"2024-08-17T02:46:16.775559+0000","last_active":"2024-08-17T02:46:16.775559+0000","last_peered":"2024-08-17T02:46:16.775559+0000","last_clean":"2024-08-17T02:46:16.775559+0000","last_became_active":"2024-08-17T02:46:16.775281+0000","last_became_peered":"2024-08-17T02:46:16.775281+0000","last_unstale":"2024-08-17T02:46:16.775559+0000","last_undegraded":"2024-08-17T02:46:16.775559+0000","last_fullsized":"2024-08-17T02:46:16.775559+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:38.566217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.785553+0000","last_change":"2024-08-17T02:46:16.777922+0000","last_active":"2024-08-17T02:46:16.785553+0000","last_peered":"2024-08-17T02:46:16.785553+0000","last_clean":"2024-08-17T02:46:16.785553+0000","last_became_active":"2024-08-17T02:46:16.777695+0000","last_became_peered":"2024-08-17T02:46:16.777695+0000","last_unstale":"2024-08-17T02:46:16.785553+0000","last_undegraded":"2024-08-17T02:46:16.785553+0000","last_fullsized":"2024-08-17T02:46:16.785553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:19:13.983272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.807299+0000","last_change":"2024-08-17T02:46:16.807299+0000","last_active":"2024-08-17T02:46:16.807299+0000","last_peered":"2024-08-17T02:46:16.807299+0000","last_clean":"2024-08-17T02:46:16.807299+0000","last_became_active":"2024-08-17T02:46:16.807104+0000","last_became_peered":"2024-08-17T02:46:16.807104+0000","last_unstale":"2024-08-17T02:46:16.807299+0000","last_undegraded":"2024-08-17T02:46:16.807299+0000","last_fullsized":"2024-08-17T02:46:16.807299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:34:45.612431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.804741+0000","last_change":"2024-08-17T02:46:16.790102+0000","last_active":"2024-08-17T02:46:16.804741+0000","last_peered":"2024-08-17T02:46:16.804741+0000","last_clean":"2024-08-17T02:46:16.804741+0000","last_became_active":"2024-08-17T02:46:16.789831+0000","last_became_peered":"2024-08-17T02:46:16.789831+0000","last_unstale":"2024-08-17T02:46:16.804741+0000","last_undegraded":"2024-08-17T02:46:16.804741+0000","last_fullsized":"2024-08-17T02:46:16.804741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:31:24.953450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.787399+0000","last_change":"2024-08-17T02:46:16.783028+0000","last_active":"2024-08-17T02:46:16.787399+0000","last_peered":"2024-08-17T02:46:16.787399+0000","last_clean":"2024-08-17T02:46:16.787399+0000","last_became_active":"2024-08-17T02:46:16.782595+0000","last_became_peered":"2024-08-17T02:46:16.782595+0000","last_unstale":"2024-08-17T02:46:16.787399+0000","last_undegraded":"2024-08-17T02:46:16.787399+0000","last_fullsized":"2024-08-17T02:46:16.787399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:25:19.636862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.804818+0000","last_change":"2024-08-17T02:46:16.801306+0000","last_active":"2024-08-17T02:46:16.804818+0000","last_peered":"2024-08-17T02:46:16.804818+0000","last_clean":"2024-08-17T02:46:16.804818+0000","last_became_active":"2024-08-17T02:46:16.800998+0000","last_became_peered":"2024-08-17T02:46:16.800998+0000","last_unstale":"2024-08-17T02:46:16.804818+0000","last_undegraded":"2024-08-17T02:46:16.804818+0000","last_fullsized":"2024-08-17T02:46:16.804818+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:44:22.048521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.773451+0000","last_change":"2024-08-17T02:46:16.773451+0000","last_active":"2024-08-17T02:46:16.773451+0000","last_peered":"2024-08-17T02:46:16.773451+0000","last_clean":"2024-08-17T02:46:16.773451+0000","last_became_active":"2024-08-17T02:46:16.773189+0000","last_became_peered":"2024-08-17T02:46:16.773189+0000","last_unstale":"2024-08-17T02:46:16.773451+0000","last_undegraded":"2024-08-17T02:46:16.773451+0000","last_fullsized":"2024-08-17T02:46:16.773451+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:39:22.034306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.805135+0000","last_change":"2024-08-17T02:46:16.797624+0000","last_active":"2024-08-17T02:46:16.805135+0000","last_peered":"2024-08-17T02:46:16.805135+0000","last_clean":"2024-08-17T02:46:16.805135+0000","last_became_active":"2024-08-17T02:46:16.790772+0000","last_became_peered":"2024-08-17T02:46:16.790772+0000","last_unstale":"2024-08-17T02:46:16.805135+0000","last_undegraded":"2024-08-17T02:46:16.805135+0000","last_fullsized":"2024-08-17T02:46:16.805135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:19:59.271493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.784253+0000","last_change":"2024-08-17T02:46:16.784253+0000","last_active":"2024-08-17T02:46:16.784253+0000","last_peered":"2024-08-17T02:46:16.784253+0000","last_clean":"2024-08-17T02:46:16.784253+0000","last_became_active":"2024-08-17T02:46:16.782422+0000","last_became_peered":"2024-08-17T02:46:16.782422+0000","last_unstale":"2024-08-17T02:46:16.784253+0000","last_undegraded":"2024-08-17T02:46:16.784253+0000","last_fullsized":"2024-08-17T02:46:16.784253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:44:22.048521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.778183+0000","last_change":"2024-08-17T02:46:16.778183+0000","last_active":"2024-08-17T02:46:16.778183+0000","last_peered":"2024-08-17T02:46:16.778183+0000","last_clean":"2024-08-17T02:46:16.778183+0000","last_became_active":"2024-08-17T02:46:16.777830+0000","last_became_peered":"2024-08-17T02:46:16.777830+0000","last_unstale":"2024-08-17T02:46:16.778183+0000","last_undegraded":"2024-08-17T02:46:16.778183+0000","last_fullsized":"2024-08-17T02:46:16.778183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:39:22.034306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.797835+0000","last_change":"2024-08-17T02:46:16.785642+0000","last_active":"2024-08-17T02:46:16.797835+0000","last_peered":"2024-08-17T02:46:16.797835+0000","last_clean":"2024-08-17T02:46:16.797835+0000","last_became_active":"2024-08-17T02:46:16.785329+0000","last_became_peered":"2024-08-17T02:46:16.785329+0000","last_unstale":"2024-08-17T02:46:16.797835+0000","last_undegraded":"2024-08-17T02:46:16.797835+0000","last_fullsized":"2024-08-17T02:46:16.797835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:59:23.376039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.793188+0000","last_change":"2024-08-17T02:46:16.793188+0000","last_active":"2024-08-17T02:46:16.793188+0000","last_peered":"2024-08-17T02:46:16.793188+0000","last_clean":"2024-08-17T02:46:16.793188+0000","last_became_active":"2024-08-17T02:46:16.792764+0000","last_became_peered":"2024-08-17T02:46:16.792764+0000","last_unstale":"2024-08-17T02:46:16.793188+0000","last_undegraded":"2024-08-17T02:46:16.793188+0000","last_fullsized":"2024-08-17T02:46:16.793188+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:45:18.517491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.779163+0000","last_change":"2024-08-17T02:46:16.779163+0000","last_active":"2024-08-17T02:46:16.779163+0000","last_peered":"2024-08-17T02:46:16.779163+0000","last_clean":"2024-08-17T02:46:16.779163+0000","last_became_active":"2024-08-17T02:46:16.778884+0000","last_became_peered":"2024-08-17T02:46:16.778884+0000","last_unstale":"2024-08-17T02:46:16.779163+0000","last_undegraded":"2024-08-17T02:46:16.779163+0000","last_fullsized":"2024-08-17T02:46:16.779163+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:38:11.703986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.803561+0000","last_change":"2024-08-17T02:46:15.778364+0000","last_active":"2024-08-17T02:46:16.803561+0000","last_peered":"2024-08-17T02:46:16.803561+0000","last_clean":"2024-08-17T02:46:16.803561+0000","last_became_active":"2024-08-17T02:46:15.778053+0000","last_became_peered":"2024-08-17T02:46:15.778053+0000","last_unstale":"2024-08-17T02:46:16.803561+0000","last_undegraded":"2024-08-17T02:46:16.803561+0000","last_fullsized":"2024-08-17T02:46:16.803561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:59:22.369433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.798087+0000","last_change":"2024-08-17T02:46:16.780991+0000","last_active":"2024-08-17T02:46:16.798087+0000","last_peered":"2024-08-17T02:46:16.798087+0000","last_clean":"2024-08-17T02:46:16.798087+0000","last_became_active":"2024-08-17T02:46:16.780479+0000","last_became_peered":"2024-08-17T02:46:16.780479+0000","last_unstale":"2024-08-17T02:46:16.798087+0000","last_undegraded":"2024-08-17T02:46:16.798087+0000","last_fullsized":"2024-08-17T02:46:16.798087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:44:38.564320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.807436+0000","last_change":"2024-08-17T02:46:15.779781+0000","last_active":"2024-08-17T02:46:16.807436+0000","last_peered":"2024-08-17T02:46:16.807436+0000","last_clean":"2024-08-17T02:46:16.807436+0000","last_became_active":"2024-08-17T02:46:15.779599+0000","last_became_peered":"2024-08-17T02:46:15.779599+0000","last_unstale":"2024-08-17T02:46:16.807436+0000","last_undegraded":"2024-08-17T02:46:16.807436+0000","last_fullsized":"2024-08-17T02:46:16.807436+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:32:57.038113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.808589+0000","last_change":"2024-08-17T02:46:16.799787+0000","last_active":"2024-08-17T02:46:16.808589+0000","last_peered":"2024-08-17T02:46:16.808589+0000","last_clean":"2024-08-17T02:46:16.808589+0000","last_became_active":"2024-08-17T02:46:16.790378+0000","last_became_peered":"2024-08-17T02:46:16.790378+0000","last_unstale":"2024-08-17T02:46:16.808589+0000","last_undegraded":"2024-08-17T02:46:16.808589+0000","last_fullsized":"2024-08-17T02:46:16.808589+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:24:25.550110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.800456+0000","last_change":"2024-08-17T02:46:15.784541+0000","last_active":"2024-08-17T02:46:16.800456+0000","last_peered":"2024-08-17T02:46:16.800456+0000","last_clean":"2024-08-17T02:46:16.800456+0000","last_became_active":"2024-08-17T02:46:15.784063+0000","last_became_peered":"2024-08-17T02:46:15.784063+0000","last_unstale":"2024-08-17T02:46:16.800456+0000","last_undegraded":"2024-08-17T02:46:16.800456+0000","last_fullsized":"2024-08-17T02:46:16.800456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:31:13.415009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.790058+0000","last_change":"2024-08-17T02:46:16.785115+0000","last_active":"2024-08-17T02:46:16.790058+0000","last_peered":"2024-08-17T02:46:16.790058+0000","last_clean":"2024-08-17T02:46:16.790058+0000","last_became_active":"2024-08-17T02:46:16.784009+0000","last_became_peered":"2024-08-17T02:46:16.784009+0000","last_unstale":"2024-08-17T02:46:16.790058+0000","last_undegraded":"2024-08-17T02:46:16.790058+0000","last_fullsized":"2024-08-17T02:46:16.790058+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:24:43.890584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.805211+0000","last_change":"2024-08-17T02:46:15.786518+0000","last_active":"2024-08-17T02:46:16.805211+0000","last_peered":"2024-08-17T02:46:16.805211+0000","last_clean":"2024-08-17T02:46:16.805211+0000","last_became_active":"2024-08-17T02:46:15.785432+0000","last_became_peered":"2024-08-17T02:46:15.785432+0000","last_unstale":"2024-08-17T02:46:16.805211+0000","last_undegraded":"2024-08-17T02:46:16.805211+0000","last_fullsized":"2024-08-17T02:46:16.805211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:43:34.809998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.800722+0000","last_change":"2024-08-17T02:46:16.795299+0000","last_active":"2024-08-17T02:46:16.800722+0000","last_peered":"2024-08-17T02:46:16.800722+0000","last_clean":"2024-08-17T02:46:16.800722+0000","last_became_active":"2024-08-17T02:46:16.787463+0000","last_became_peered":"2024-08-17T02:46:16.787463+0000","last_unstale":"2024-08-17T02:46:16.800722+0000","last_undegraded":"2024-08-17T02:46:16.800722+0000","last_fullsized":"2024-08-17T02:46:16.800722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:04.740084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.812938+0000","last_change":"2024-08-17T02:46:15.776329+0000","last_active":"2024-08-17T02:46:16.812938+0000","last_peered":"2024-08-17T02:46:16.812938+0000","last_clean":"2024-08-17T02:46:16.812938+0000","last_became_active":"2024-08-17T02:46:15.774914+0000","last_became_peered":"2024-08-17T02:46:15.774914+0000","last_unstale":"2024-08-17T02:46:16.812938+0000","last_undegraded":"2024-08-17T02:46:16.812938+0000","last_fullsized":"2024-08-17T02:46:16.812938+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:37:24.295411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.794683+0000","last_change":"2024-08-17T02:46:16.784692+0000","last_active":"2024-08-17T02:46:16.794683+0000","last_peered":"2024-08-17T02:46:16.794683+0000","last_clean":"2024-08-17T02:46:16.794683+0000","last_became_active":"2024-08-17T02:46:16.784443+0000","last_became_peered":"2024-08-17T02:46:16.784443+0000","last_unstale":"2024-08-17T02:46:16.794683+0000","last_undegraded":"2024-08-17T02:46:16.794683+0000","last_fullsized":"2024-08-17T02:46:16.794683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:38.566217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.810145+0000","last_change":"2024-08-17T02:46:15.783239+0000","last_active":"2024-08-17T02:46:16.810145+0000","last_peered":"2024-08-17T02:46:16.810145+0000","last_clean":"2024-08-17T02:46:16.810145+0000","last_became_active":"2024-08-17T02:46:15.782992+0000","last_became_peered":"2024-08-17T02:46:15.782992+0000","last_unstale":"2024-08-17T02:46:16.810145+0000","last_undegraded":"2024-08-17T02:46:16.810145+0000","last_fullsized":"2024-08-17T02:46:16.810145+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:32:57.038113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.796815+0000","last_change":"2024-08-17T02:46:16.794186+0000","last_active":"2024-08-17T02:46:16.796815+0000","last_peered":"2024-08-17T02:46:16.796815+0000","last_clean":"2024-08-17T02:46:16.796815+0000","last_became_active":"2024-08-17T02:46:16.793863+0000","last_became_peered":"2024-08-17T02:46:16.793863+0000","last_unstale":"2024-08-17T02:46:16.796815+0000","last_undegraded":"2024-08-17T02:46:16.796815+0000","last_fullsized":"2024-08-17T02:46:16.796815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:36:20.309762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.776653+0000","last_change":"2024-08-17T02:46:15.785254+0000","last_active":"2024-08-17T02:46:16.776653+0000","last_peered":"2024-08-17T02:46:16.776653+0000","last_clean":"2024-08-17T02:46:16.776653+0000","last_became_active":"2024-08-17T02:46:15.784911+0000","last_became_peered":"2024-08-17T02:46:15.784911+0000","last_unstale":"2024-08-17T02:46:16.776653+0000","last_undegraded":"2024-08-17T02:46:16.776653+0000","last_fullsized":"2024-08-17T02:46:16.776653+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:17:51.619358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.785567+0000","last_change":"2024-08-17T02:46:16.781047+0000","last_active":"2024-08-17T02:46:16.785567+0000","last_peered":"2024-08-17T02:46:16.785567+0000","last_clean":"2024-08-17T02:46:16.785567+0000","last_became_active":"2024-08-17T02:46:16.780772+0000","last_became_peered":"2024-08-17T02:46:16.780772+0000","last_unstale":"2024-08-17T02:46:16.785567+0000","last_undegraded":"2024-08-17T02:46:16.785567+0000","last_fullsized":"2024-08-17T02:46:16.785567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:16:59.558851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.766983+0000","last_change":"2024-08-17T02:46:15.779695+0000","last_active":"2024-08-17T02:46:16.766983+0000","last_peered":"2024-08-17T02:46:16.766983+0000","last_clean":"2024-08-17T02:46:16.766983+0000","last_became_active":"2024-08-17T02:46:15.779452+0000","last_became_peered":"2024-08-17T02:46:15.779452+0000","last_unstale":"2024-08-17T02:46:16.766983+0000","last_undegraded":"2024-08-17T02:46:16.766983+0000","last_fullsized":"2024-08-17T02:46:16.766983+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:31:49.347928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.775032+0000","last_change":"2024-08-17T02:46:16.775032+0000","last_active":"2024-08-17T02:46:16.775032+0000","last_peered":"2024-08-17T02:46:16.775032+0000","last_clean":"2024-08-17T02:46:16.775032+0000","last_became_active":"2024-08-17T02:46:16.774773+0000","last_became_peered":"2024-08-17T02:46:16.774773+0000","last_unstale":"2024-08-17T02:46:16.775032+0000","last_undegraded":"2024-08-17T02:46:16.775032+0000","last_fullsized":"2024-08-17T02:46:16.775032+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:02:03.656421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.786914+0000","last_change":"2024-08-17T02:46:15.782088+0000","last_active":"2024-08-17T02:46:15.786914+0000","last_peered":"2024-08-17T02:46:15.786914+0000","last_clean":"2024-08-17T02:46:15.786914+0000","last_became_active":"2024-08-17T02:46:15.781896+0000","last_became_peered":"2024-08-17T02:46:15.781896+0000","last_unstale":"2024-08-17T02:46:15.786914+0000","last_undegraded":"2024-08-17T02:46:15.786914+0000","last_fullsized":"2024-08-17T02:46:15.786914+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:43:34.809998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.787380+0000","last_change":"2024-08-17T02:46:16.782862+0000","last_active":"2024-08-17T02:46:16.787380+0000","last_peered":"2024-08-17T02:46:16.787380+0000","last_clean":"2024-08-17T02:46:16.787380+0000","last_became_active":"2024-08-17T02:46:16.782332+0000","last_became_peered":"2024-08-17T02:46:16.782332+0000","last_unstale":"2024-08-17T02:46:16.787380+0000","last_undegraded":"2024-08-17T02:46:16.787380+0000","last_fullsized":"2024-08-17T02:46:16.787380+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:03:38.566217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.780347+0000","last_change":"2024-08-17T02:46:15.774480+0000","last_active":"2024-08-17T02:46:15.780347+0000","last_peered":"2024-08-17T02:46:15.780347+0000","last_clean":"2024-08-17T02:46:15.780347+0000","last_became_active":"2024-08-17T02:46:15.774193+0000","last_became_peered":"2024-08-17T02:46:15.774193+0000","last_unstale":"2024-08-17T02:46:15.780347+0000","last_undegraded":"2024-08-17T02:46:15.780347+0000","last_fullsized":"2024-08-17T02:46:15.780347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:35:16.966461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.790910+0000","last_change":"2024-08-17T02:46:16.787330+0000","last_active":"2024-08-17T02:46:16.790910+0000","last_peered":"2024-08-17T02:46:16.790910+0000","last_clean":"2024-08-17T02:46:16.790910+0000","last_became_active":"2024-08-17T02:46:16.787060+0000","last_became_peered":"2024-08-17T02:46:16.787060+0000","last_unstale":"2024-08-17T02:46:16.790910+0000","last_undegraded":"2024-08-17T02:46:16.790910+0000","last_fullsized":"2024-08-17T02:46:16.790910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:19:13.983272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.802472+0000","last_change":"2024-08-17T02:46:15.778373+0000","last_active":"2024-08-17T02:46:16.802472+0000","last_peered":"2024-08-17T02:46:16.802472+0000","last_clean":"2024-08-17T02:46:16.802472+0000","last_became_active":"2024-08-17T02:46:15.778127+0000","last_became_peered":"2024-08-17T02:46:15.778127+0000","last_unstale":"2024-08-17T02:46:16.802472+0000","last_undegraded":"2024-08-17T02:46:16.802472+0000","last_fullsized":"2024-08-17T02:46:16.802472+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:06:27.501528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.786778+0000","last_change":"2024-08-17T02:46:16.786778+0000","last_active":"2024-08-17T02:46:16.786778+0000","last_peered":"2024-08-17T02:46:16.786778+0000","last_clean":"2024-08-17T02:46:16.786778+0000","last_became_active":"2024-08-17T02:46:16.786393+0000","last_became_peered":"2024-08-17T02:46:16.786393+0000","last_unstale":"2024-08-17T02:46:16.786778+0000","last_undegraded":"2024-08-17T02:46:16.786778+0000","last_fullsized":"2024-08-17T02:46:16.786778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:02:15.765459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.805328+0000","last_change":"2024-08-17T02:46:15.785147+0000","last_active":"2024-08-17T02:46:16.805328+0000","last_peered":"2024-08-17T02:46:16.805328+0000","last_clean":"2024-08-17T02:46:16.805328+0000","last_became_active":"2024-08-17T02:46:15.784855+0000","last_became_peered":"2024-08-17T02:46:15.784855+0000","last_unstale":"2024-08-17T02:46:16.805328+0000","last_undegraded":"2024-08-17T02:46:16.805328+0000","last_fullsized":"2024-08-17T02:46:16.805328+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:45:56.402053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.789376+0000","last_change":"2024-08-17T02:46:16.789376+0000","last_active":"2024-08-17T02:46:16.789376+0000","last_peered":"2024-08-17T02:46:16.789376+0000","last_clean":"2024-08-17T02:46:16.789376+0000","last_became_active":"2024-08-17T02:46:16.786740+0000","last_became_peered":"2024-08-17T02:46:16.786740+0000","last_unstale":"2024-08-17T02:46:16.789376+0000","last_undegraded":"2024-08-17T02:46:16.789376+0000","last_fullsized":"2024-08-17T02:46:16.789376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:36:44.763322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.810160+0000","last_change":"2024-08-17T02:46:15.776876+0000","last_active":"2024-08-17T02:46:16.810160+0000","last_peered":"2024-08-17T02:46:16.810160+0000","last_clean":"2024-08-17T02:46:16.810160+0000","last_became_active":"2024-08-17T02:46:15.775681+0000","last_became_peered":"2024-08-17T02:46:15.775681+0000","last_unstale":"2024-08-17T02:46:16.810160+0000","last_undegraded":"2024-08-17T02:46:16.810160+0000","last_fullsized":"2024-08-17T02:46:16.810160+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:02:59.926303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.788132+0000","last_change":"2024-08-17T02:46:16.784377+0000","last_active":"2024-08-17T02:46:16.788132+0000","last_peered":"2024-08-17T02:46:16.788132+0000","last_clean":"2024-08-17T02:46:16.788132+0000","last_became_active":"2024-08-17T02:46:16.784056+0000","last_became_peered":"2024-08-17T02:46:16.784056+0000","last_unstale":"2024-08-17T02:46:16.788132+0000","last_undegraded":"2024-08-17T02:46:16.788132+0000","last_fullsized":"2024-08-17T02:46:16.788132+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:31:24.953450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.777168+0000","last_change":"2024-08-17T02:46:15.778765+0000","last_active":"2024-08-17T02:46:16.777168+0000","last_peered":"2024-08-17T02:46:16.777168+0000","last_clean":"2024-08-17T02:46:16.777168+0000","last_became_active":"2024-08-17T02:46:15.778410+0000","last_became_peered":"2024-08-17T02:46:15.778410+0000","last_unstale":"2024-08-17T02:46:16.777168+0000","last_undegraded":"2024-08-17T02:46:16.777168+0000","last_fullsized":"2024-08-17T02:46:16.777168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:02:59.926303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.809172+0000","last_change":"2024-08-17T02:46:16.808197+0000","last_active":"2024-08-17T02:46:16.809172+0000","last_peered":"2024-08-17T02:46:16.809172+0000","last_clean":"2024-08-17T02:46:16.809172+0000","last_became_active":"2024-08-17T02:46:16.804880+0000","last_became_peered":"2024-08-17T02:46:16.804880+0000","last_unstale":"2024-08-17T02:46:16.809172+0000","last_undegraded":"2024-08-17T02:46:16.809172+0000","last_fullsized":"2024-08-17T02:46:16.809172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:34:26.609150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.776979+0000","last_change":"2024-08-17T02:46:15.780068+0000","last_active":"2024-08-17T02:46:16.776979+0000","last_peered":"2024-08-17T02:46:16.776979+0000","last_clean":"2024-08-17T02:46:16.776979+0000","last_became_active":"2024-08-17T02:46:15.778500+0000","last_became_peered":"2024-08-17T02:46:15.778500+0000","last_unstale":"2024-08-17T02:46:16.776979+0000","last_undegraded":"2024-08-17T02:46:16.776979+0000","last_fullsized":"2024-08-17T02:46:16.776979+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:51:10.218325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.781865+0000","last_change":"2024-08-17T02:46:16.781865+0000","last_active":"2024-08-17T02:46:16.781865+0000","last_peered":"2024-08-17T02:46:16.781865+0000","last_clean":"2024-08-17T02:46:16.781865+0000","last_became_active":"2024-08-17T02:46:16.778795+0000","last_became_peered":"2024-08-17T02:46:16.778795+0000","last_unstale":"2024-08-17T02:46:16.781865+0000","last_undegraded":"2024-08-17T02:46:16.781865+0000","last_fullsized":"2024-08-17T02:46:16.781865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:34:20.741425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.803935+0000","last_change":"2024-08-17T02:46:15.785957+0000","last_active":"2024-08-17T02:46:16.803935+0000","last_peered":"2024-08-17T02:46:16.803935+0000","last_clean":"2024-08-17T02:46:16.803935+0000","last_became_active":"2024-08-17T02:46:15.785692+0000","last_became_peered":"2024-08-17T02:46:15.785692+0000","last_unstale":"2024-08-17T02:46:16.803935+0000","last_undegraded":"2024-08-17T02:46:16.803935+0000","last_fullsized":"2024-08-17T02:46:16.803935+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:32:43.979148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.786826+0000","last_change":"2024-08-17T02:46:16.786826+0000","last_active":"2024-08-17T02:46:16.786826+0000","last_peered":"2024-08-17T02:46:16.786826+0000","last_clean":"2024-08-17T02:46:16.786826+0000","last_became_active":"2024-08-17T02:46:16.785671+0000","last_became_peered":"2024-08-17T02:46:16.785671+0000","last_unstale":"2024-08-17T02:46:16.786826+0000","last_undegraded":"2024-08-17T02:46:16.786826+0000","last_fullsized":"2024-08-17T02:46:16.786826+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T07:53:59.611294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.802779+0000","last_change":"2024-08-17T02:46:15.784448+0000","last_active":"2024-08-17T02:46:16.802779+0000","last_peered":"2024-08-17T02:46:16.802779+0000","last_clean":"2024-08-17T02:46:16.802779+0000","last_became_active":"2024-08-17T02:46:15.784156+0000","last_became_peered":"2024-08-17T02:46:15.784156+0000","last_unstale":"2024-08-17T02:46:16.802779+0000","last_undegraded":"2024-08-17T02:46:16.802779+0000","last_fullsized":"2024-08-17T02:46:16.802779+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:27:14.740630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.806807+0000","last_change":"2024-08-17T02:46:16.796802+0000","last_active":"2024-08-17T02:46:16.806807+0000","last_peered":"2024-08-17T02:46:16.806807+0000","last_clean":"2024-08-17T02:46:16.806807+0000","last_became_active":"2024-08-17T02:46:16.796547+0000","last_became_peered":"2024-08-17T02:46:16.796547+0000","last_unstale":"2024-08-17T02:46:16.806807+0000","last_undegraded":"2024-08-17T02:46:16.806807+0000","last_fullsized":"2024-08-17T02:46:16.806807+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:47:09.835864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.808984+0000","last_change":"2024-08-17T02:46:15.776482+0000","last_active":"2024-08-17T02:46:16.808984+0000","last_peered":"2024-08-17T02:46:16.808984+0000","last_clean":"2024-08-17T02:46:16.808984+0000","last_became_active":"2024-08-17T02:46:15.775200+0000","last_became_peered":"2024-08-17T02:46:15.775200+0000","last_unstale":"2024-08-17T02:46:16.808984+0000","last_undegraded":"2024-08-17T02:46:16.808984+0000","last_fullsized":"2024-08-17T02:46:16.808984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:47:08.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.808041+0000","last_change":"2024-08-17T02:46:16.808041+0000","last_active":"2024-08-17T02:46:16.808041+0000","last_peered":"2024-08-17T02:46:16.808041+0000","last_clean":"2024-08-17T02:46:16.808041+0000","last_became_active":"2024-08-17T02:46:16.807798+0000","last_became_peered":"2024-08-17T02:46:16.807798+0000","last_unstale":"2024-08-17T02:46:16.808041+0000","last_undegraded":"2024-08-17T02:46:16.808041+0000","last_fullsized":"2024-08-17T02:46:16.808041+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:59:23.376039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.772244+0000","last_change":"2024-08-17T02:46:15.786125+0000","last_active":"2024-08-17T02:46:16.772244+0000","last_peered":"2024-08-17T02:46:16.772244+0000","last_clean":"2024-08-17T02:46:16.772244+0000","last_became_active":"2024-08-17T02:46:15.785926+0000","last_became_peered":"2024-08-17T02:46:15.785926+0000","last_unstale":"2024-08-17T02:46:16.772244+0000","last_undegraded":"2024-08-17T02:46:16.772244+0000","last_fullsized":"2024-08-17T02:46:16.772244+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:44:37.557714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.786226+0000","last_change":"2024-08-17T02:46:16.782084+0000","last_active":"2024-08-17T02:46:16.786226+0000","last_peered":"2024-08-17T02:46:16.786226+0000","last_clean":"2024-08-17T02:46:16.786226+0000","last_became_active":"2024-08-17T02:46:16.781650+0000","last_became_peered":"2024-08-17T02:46:16.781650+0000","last_unstale":"2024-08-17T02:46:16.786226+0000","last_undegraded":"2024-08-17T02:46:16.786226+0000","last_fullsized":"2024-08-17T02:46:16.786226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:51:11.224931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.786106+0000","last_change":"2024-08-17T02:46:15.784816+0000","last_active":"2024-08-17T02:46:15.786106+0000","last_peered":"2024-08-17T02:46:15.786106+0000","last_clean":"2024-08-17T02:46:15.786106+0000","last_became_active":"2024-08-17T02:46:15.784489+0000","last_became_peered":"2024-08-17T02:46:15.784489+0000","last_unstale":"2024-08-17T02:46:15.786106+0000","last_undegraded":"2024-08-17T02:46:15.786106+0000","last_fullsized":"2024-08-17T02:46:15.786106+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:37:24.295411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.778727+0000","last_change":"2024-08-17T02:46:16.778727+0000","last_active":"2024-08-17T02:46:16.778727+0000","last_peered":"2024-08-17T02:46:16.778727+0000","last_clean":"2024-08-17T02:46:16.778727+0000","last_became_active":"2024-08-17T02:46:16.778387+0000","last_became_peered":"2024-08-17T02:46:16.778387+0000","last_unstale":"2024-08-17T02:46:16.778727+0000","last_undegraded":"2024-08-17T02:46:16.778727+0000","last_fullsized":"2024-08-17T02:46:16.778727+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:13:16.818070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.802526+0000","last_change":"2024-08-17T02:46:15.774187+0000","last_active":"2024-08-17T02:46:16.802526+0000","last_peered":"2024-08-17T02:46:16.802526+0000","last_clean":"2024-08-17T02:46:16.802526+0000","last_became_active":"2024-08-17T02:46:15.773919+0000","last_became_peered":"2024-08-17T02:46:15.773919+0000","last_unstale":"2024-08-17T02:46:16.802526+0000","last_undegraded":"2024-08-17T02:46:16.802526+0000","last_fullsized":"2024-08-17T02:46:16.802526+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:37:24.295411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.777955+0000","last_change":"2024-08-17T02:46:16.777955+0000","last_active":"2024-08-17T02:46:16.777955+0000","last_peered":"2024-08-17T02:46:16.777955+0000","last_clean":"2024-08-17T02:46:16.777955+0000","last_became_active":"2024-08-17T02:46:16.777500+0000","last_became_peered":"2024-08-17T02:46:16.777500+0000","last_unstale":"2024-08-17T02:46:16.777955+0000","last_undegraded":"2024-08-17T02:46:16.777955+0000","last_fullsized":"2024-08-17T02:46:16.777955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:02:07.541310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.766201+0000","last_change":"2024-08-17T02:46:15.775398+0000","last_active":"2024-08-17T02:46:16.766201+0000","last_peered":"2024-08-17T02:46:16.766201+0000","last_clean":"2024-08-17T02:46:16.766201+0000","last_became_active":"2024-08-17T02:46:15.775106+0000","last_became_peered":"2024-08-17T02:46:15.775106+0000","last_unstale":"2024-08-17T02:46:16.766201+0000","last_undegraded":"2024-08-17T02:46:16.766201+0000","last_fullsized":"2024-08-17T02:46:16.766201+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:51:10.218325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.787186+0000","last_change":"2024-08-17T02:46:16.787186+0000","last_active":"2024-08-17T02:46:16.787186+0000","last_peered":"2024-08-17T02:46:16.787186+0000","last_clean":"2024-08-17T02:46:16.787186+0000","last_became_active":"2024-08-17T02:46:16.786893+0000","last_became_peered":"2024-08-17T02:46:16.786893+0000","last_unstale":"2024-08-17T02:46:16.787186+0000","last_undegraded":"2024-08-17T02:46:16.787186+0000","last_fullsized":"2024-08-17T02:46:16.787186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T03:39:22.034306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.797571+0000","last_change":"2024-08-17T02:46:15.779121+0000","last_active":"2024-08-17T02:46:16.797571+0000","last_peered":"2024-08-17T02:46:16.797571+0000","last_clean":"2024-08-17T02:46:16.797571+0000","last_became_active":"2024-08-17T02:46:15.778318+0000","last_became_peered":"2024-08-17T02:46:15.778318+0000","last_unstale":"2024-08-17T02:46:16.797571+0000","last_undegraded":"2024-08-17T02:46:16.797571+0000","last_fullsized":"2024-08-17T02:46:16.797571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:38:54.397608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.806933+0000","last_change":"2024-08-17T02:46:16.800899+0000","last_active":"2024-08-17T02:46:16.806933+0000","last_peered":"2024-08-17T02:46:16.806933+0000","last_clean":"2024-08-17T02:46:16.806933+0000","last_became_active":"2024-08-17T02:46:16.798624+0000","last_became_peered":"2024-08-17T02:46:16.798624+0000","last_unstale":"2024-08-17T02:46:16.806933+0000","last_undegraded":"2024-08-17T02:46:16.806933+0000","last_fullsized":"2024-08-17T02:46:16.806933+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:27:15.747236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.786322+0000","last_change":"2024-08-17T02:46:15.783352+0000","last_active":"2024-08-17T02:46:15.786322+0000","last_peered":"2024-08-17T02:46:15.786322+0000","last_clean":"2024-08-17T02:46:15.786322+0000","last_became_active":"2024-08-17T02:46:15.783075+0000","last_became_peered":"2024-08-17T02:46:15.783075+0000","last_unstale":"2024-08-17T02:46:15.786322+0000","last_undegraded":"2024-08-17T02:46:15.786322+0000","last_fullsized":"2024-08-17T02:46:15.786322+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:32:00.682781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.797310+0000","last_change":"2024-08-17T02:46:16.794091+0000","last_active":"2024-08-17T02:46:16.797310+0000","last_peered":"2024-08-17T02:46:16.797310+0000","last_clean":"2024-08-17T02:46:16.797310+0000","last_became_active":"2024-08-17T02:46:16.793580+0000","last_became_peered":"2024-08-17T02:46:16.793580+0000","last_unstale":"2024-08-17T02:46:16.797310+0000","last_undegraded":"2024-08-17T02:46:16.797310+0000","last_fullsized":"2024-08-17T02:46:16.797310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:32:58.044719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":11,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.803238+0000","last_change":"2024-08-17T02:46:15.781177+0000","last_active":"2024-08-17T02:46:16.803238+0000","last_peered":"2024-08-17T02:46:16.803238+0000","last_clean":"2024-08-17T02:46:16.803238+0000","last_became_active":"2024-08-17T02:46:15.780921+0000","last_became_peered":"2024-08-17T02:46:15.780921+0000","last_unstale":"2024-08-17T02:46:16.803238+0000","last_undegraded":"2024-08-17T02:46:16.803238+0000","last_fullsized":"2024-08-17T02:46:16.803238+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:31:49.347928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.782404+0000","last_change":"2024-08-17T02:46:16.782404+0000","last_active":"2024-08-17T02:46:16.782404+0000","last_peered":"2024-08-17T02:46:16.782404+0000","last_clean":"2024-08-17T02:46:16.782404+0000","last_became_active":"2024-08-17T02:46:16.781487+0000","last_became_peered":"2024-08-17T02:46:16.781487+0000","last_unstale":"2024-08-17T02:46:16.782404+0000","last_undegraded":"2024-08-17T02:46:16.782404+0000","last_fullsized":"2024-08-17T02:46:16.782404+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:12:01.932427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.783484+0000","last_change":"2024-08-17T02:46:15.776778+0000","last_active":"2024-08-17T02:46:15.783484+0000","last_peered":"2024-08-17T02:46:15.783484+0000","last_clean":"2024-08-17T02:46:15.783484+0000","last_became_active":"2024-08-17T02:46:15.776448+0000","last_became_peered":"2024-08-17T02:46:15.776448+0000","last_unstale":"2024-08-17T02:46:15.783484+0000","last_undegraded":"2024-08-17T02:46:15.783484+0000","last_fullsized":"2024-08-17T02:46:15.783484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:55:59.916719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.791248+0000","last_change":"2024-08-17T02:46:16.787985+0000","last_active":"2024-08-17T02:46:16.791248+0000","last_peered":"2024-08-17T02:46:16.791248+0000","last_clean":"2024-08-17T02:46:16.791248+0000","last_became_active":"2024-08-17T02:46:16.787668+0000","last_became_peered":"2024-08-17T02:46:16.787668+0000","last_unstale":"2024-08-17T02:46:16.791248+0000","last_undegraded":"2024-08-17T02:46:16.791248+0000","last_fullsized":"2024-08-17T02:46:16.791248+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:37:25.302017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.784183+0000","last_change":"2024-08-17T02:46:15.777081+0000","last_active":"2024-08-17T02:46:15.784183+0000","last_peered":"2024-08-17T02:46:15.784183+0000","last_clean":"2024-08-17T02:46:15.784183+0000","last_became_active":"2024-08-17T02:46:15.776804+0000","last_became_peered":"2024-08-17T02:46:15.776804+0000","last_unstale":"2024-08-17T02:46:15.784183+0000","last_undegraded":"2024-08-17T02:46:15.784183+0000","last_fullsized":"2024-08-17T02:46:15.784183+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T12:43:13.086847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.789235+0000","last_change":"2024-08-17T02:46:16.789235+0000","last_active":"2024-08-17T02:46:16.789235+0000","last_peered":"2024-08-17T02:46:16.789235+0000","last_clean":"2024-08-17T02:46:16.789235+0000","last_became_active":"2024-08-17T02:46:16.782691+0000","last_became_peered":"2024-08-17T02:46:16.782691+0000","last_unstale":"2024-08-17T02:46:16.789235+0000","last_undegraded":"2024-08-17T02:46:16.789235+0000","last_fullsized":"2024-08-17T02:46:16.789235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T11:32:01.689387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.784252+0000","last_change":"2024-08-17T02:46:15.780992+0000","last_active":"2024-08-17T02:46:15.784252+0000","last_peered":"2024-08-17T02:46:15.784252+0000","last_clean":"2024-08-17T02:46:15.784252+0000","last_became_active":"2024-08-17T02:46:15.780752+0000","last_became_peered":"2024-08-17T02:46:15.780752+0000","last_unstale":"2024-08-17T02:46:15.784252+0000","last_undegraded":"2024-08-17T02:46:15.784252+0000","last_fullsized":"2024-08-17T02:46:15.784252+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T02:47:08.829259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.788115+0000","last_change":"2024-08-17T02:46:16.783355+0000","last_active":"2024-08-17T02:46:16.788115+0000","last_peered":"2024-08-17T02:46:16.788115+0000","last_clean":"2024-08-17T02:46:16.788115+0000","last_became_active":"2024-08-17T02:46:16.782953+0000","last_became_peered":"2024-08-17T02:46:16.782953+0000","last_unstale":"2024-08-17T02:46:16.788115+0000","last_undegraded":"2024-08-17T02:46:16.788115+0000","last_fullsized":"2024-08-17T02:46:16.788115+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T05:34:26.609150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.786791+0000","last_change":"2024-08-17T02:46:15.782313+0000","last_active":"2024-08-17T02:46:15.786791+0000","last_peered":"2024-08-17T02:46:15.786791+0000","last_clean":"2024-08-17T02:46:15.786791+0000","last_became_active":"2024-08-17T02:46:15.782068+0000","last_became_peered":"2024-08-17T02:46:15.782068+0000","last_unstale":"2024-08-17T02:46:15.786791+0000","last_undegraded":"2024-08-17T02:46:15.786791+0000","last_fullsized":"2024-08-17T02:46:15.786791+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:10:03.733478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.791773+0000","last_change":"2024-08-17T02:46:16.790570+0000","last_active":"2024-08-17T02:46:16.791773+0000","last_peered":"2024-08-17T02:46:16.791773+0000","last_clean":"2024-08-17T02:46:16.791773+0000","last_became_active":"2024-08-17T02:46:16.790301+0000","last_became_peered":"2024-08-17T02:46:16.790301+0000","last_unstale":"2024-08-17T02:46:16.791773+0000","last_undegraded":"2024-08-17T02:46:16.791773+0000","last_fullsized":"2024-08-17T02:46:16.791773+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T10:24:25.550110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.766592+0000","last_change":"2024-08-17T02:46:15.779038+0000","last_active":"2024-08-17T02:46:16.766592+0000","last_peered":"2024-08-17T02:46:16.766592+0000","last_clean":"2024-08-17T02:46:16.766592+0000","last_became_active":"2024-08-17T02:46:15.775267+0000","last_became_peered":"2024-08-17T02:46:15.775267+0000","last_unstale":"2024-08-17T02:46:16.766592+0000","last_undegraded":"2024-08-17T02:46:16.766592+0000","last_fullsized":"2024-08-17T02:46:16.766592+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:43:34.809998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.783698+0000","last_change":"2024-08-17T02:46:16.783698+0000","last_active":"2024-08-17T02:46:16.783698+0000","last_peered":"2024-08-17T02:46:16.783698+0000","last_clean":"2024-08-17T02:46:16.783698+0000","last_became_active":"2024-08-17T02:46:16.781902+0000","last_became_peered":"2024-08-17T02:46:16.781902+0000","last_unstale":"2024-08-17T02:46:16.783698+0000","last_undegraded":"2024-08-17T02:46:16.783698+0000","last_fullsized":"2024-08-17T02:46:16.783698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:24:54.295510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.776850+0000","last_change":"2024-08-17T02:46:15.780976+0000","last_active":"2024-08-17T02:46:16.776850+0000","last_peered":"2024-08-17T02:46:16.776850+0000","last_clean":"2024-08-17T02:46:16.776850+0000","last_became_active":"2024-08-17T02:46:15.780616+0000","last_became_peered":"2024-08-17T02:46:15.780616+0000","last_unstale":"2024-08-17T02:46:16.776850+0000","last_undegraded":"2024-08-17T02:46:16.776850+0000","last_fullsized":"2024-08-17T02:46:16.776850+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T04:37:24.295411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.787791+0000","last_change":"2024-08-17T02:46:16.787791+0000","last_active":"2024-08-17T02:46:16.787791+0000","last_peered":"2024-08-17T02:46:16.787791+0000","last_clean":"2024-08-17T02:46:16.787791+0000","last_became_active":"2024-08-17T02:46:16.787571+0000","last_became_peered":"2024-08-17T02:46:16.787571+0000","last_unstale":"2024-08-17T02:46:16.787791+0000","last_undegraded":"2024-08-17T02:46:16.787791+0000","last_fullsized":"2024-08-17T02:46:16.787791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:15.749133+0000","last_clean_scrub_stamp":"2024-08-17T02:46:15.749133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T08:19:59.271493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-08-17T02:46:15.787137+0000","last_change":"2024-08-17T02:46:15.785899+0000","last_active":"2024-08-17T02:46:15.787137+0000","last_peered":"2024-08-17T02:46:15.787137+0000","last_clean":"2024-08-17T02:46:15.787137+0000","last_became_active":"2024-08-17T02:46:15.785632+0000","last_became_peered":"2024-08-17T02:46:15.785632+0000","last_unstale":"2024-08-17T02:46:15.787137+0000","last_undegraded":"2024-08-17T02:46:15.787137+0000","last_fullsized":"2024-08-17T02:46:15.787137+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:46:14.742527+0000","last_clean_scrub_stamp":"2024-08-17T02:46:14.742527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T09:44:21.041915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-08-17T02:46:16.793162+0000","last_change":"2024-08-17T02:45:16.001117+0000","last_active":"2024-08-17T02:46:16.793162+0000","last_peered":"2024-08-17T02:46:16.793162+0000","last_clean":"2024-08-17T02:46:16.793162+0000","last_became_active":"2024-08-17T02:45:16.000788+0000","last_became_peered":"2024-08-17T02:45:16.000788+0000","last_unstale":"2024-08-17T02:46:16.793162+0000","last_undegraded":"2024-08-17T02:46:16.793162+0000","last_fullsized":"2024-08-17T02:46:16.793162+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_clean_scrub_stamp":"2024-08-17T02:45:14.986707+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:27:37.295374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297720,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074120,"statfs":{"total":96636764160,"available":96331898880,"internally_reserved":0,"allocated":364544,"data_stored":137418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.8780000000000001,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55700000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.8769999999999998,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.85899999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":4.4530000000000003,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":4.6870000000000003,"5min":4.6870000000000003,"15min":4.6870000000000003},"last":0.878}]},{"osd":3,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":2.798,"5min":2.798,"15min":2.798},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.66800000000000004}]},{"osd":4,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80000000000000004}]},{"osd":5,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.91900000000000004}]},{"osd":6,"last update":"Sat Aug 17 02:46:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.753},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.70899999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297724,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074116,"statfs":{"total":96636764160,"available":96331894784,"internally_reserved":0,"allocated":372736,"data_stored":138948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.47699999999999998}]},{"osd":1,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.52000000000000002}]},{"osd":2,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70999999999999996}]},{"osd":3,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":4.0570000000000004,"5min":4.0570000000000004,"15min":4.0570000000000004},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.86399999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.80300000000000005}]},{"osd":7,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.748},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":1.405}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073544,"statfs":{"total":96636764160,"available":96331309056,"internally_reserved":0,"allocated":958464,"data_stored":728334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.80100000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.71199999999999997}]},{"osd":2,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72399999999999998}]},{"osd":3,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.84899999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.496},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55300000000000005}]},{"osd":6,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59699999999999998}]},{"osd":7,"last update":"Sat Aug 17 02:46:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.73999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297736,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074104,"statfs":{"total":96636764160,"available":96331882496,"internally_reserved":0,"allocated":389120,"data_stored":145258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.78000000000000003}]},{"osd":1,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.74399999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.623},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.76400000000000001}]},{"osd":5,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84199999999999997}]},{"osd":6,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.81200000000000006}]},{"osd":7,"last update":"Sat Aug 17 02:46:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.72499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301892,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069948,"statfs":{"total":96636764160,"available":96327626752,"internally_reserved":0,"allocated":376832,"data_stored":138160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.311},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.552,"5min":3.552,"15min":3.552},"last":0.41799999999999998}]},{"osd":1,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.44}]},{"osd":2,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.8149999999999999,"5min":3.8149999999999999,"15min":3.8149999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":0.48699999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":3.9009999999999998,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":4.3360000000000003,"5min":4.3360000000000003,"15min":4.3360000000000003},"last":0.85199999999999998}]},{"osd":5,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":3.371,"5min":3.371,"15min":3.371},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":3.8599999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.93400000000000005}]},{"osd":6,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":4.4160000000000004,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":0.91200000000000003}]},{"osd":7,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":3.931,"5min":3.931,"15min":3.931},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":4.3799999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.89900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301880,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069960,"statfs":{"total":96636764160,"available":96327639040,"internally_reserved":0,"allocated":364544,"data_stored":137430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.44700000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.373},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.48199999999999998}]},{"osd":3,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":7.5960000000000001,"5min":7.5960000000000001,"15min":7.5960000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":6.0499999999999998,"5min":6.0499999999999998,"15min":6.0499999999999998},"last":0.53600000000000003}]},{"osd":4,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":5.7009999999999996,"5min":5.7009999999999996,"15min":5.7009999999999996},"last":0.73699999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":7.5220000000000002,"5min":7.5220000000000002,"15min":7.5220000000000002},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":5.968,"5min":5.968,"15min":5.968},"last":0.60599999999999998}]},{"osd":6,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":5.8780000000000001,"5min":5.8780000000000001,"15min":5.8780000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":7.5570000000000004,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.63800000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:46:13 2024","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":5.9119999999999999,"5min":5.9119999999999999,"15min":5.9119999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":7.4800000000000004,"5min":7.4800000000000004,"15min":7.4800000000000004},"last":0.71199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":970752,"data_stored":734122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.34599999999999997}]},{"osd":2,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.45400000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.495},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.9399999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.41499999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.71499999999999997}]},{"osd":6,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":2.5059999999999998,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":0.69499999999999995}]},{"osd":7,"last update":"Sat Aug 17 02:46:17 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":2.577,"5min":2.577,"15min":2.577},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.75600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301880,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069960,"statfs":{"total":96636764160,"available":96327639040,"internally_reserved":0,"allocated":364544,"data_stored":137486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.39700000000000002}]},{"osd":2,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.47999999999999998}]},{"osd":3,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.53700000000000003}]},{"osd":4,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.747},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.82899999999999996}]},{"osd":5,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.85599999999999998}]},{"osd":7,"last update":"Sat Aug 17 02:46:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.874}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:46:21.416 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-08-17T02:46:21.416 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-08-17T02:46:21.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.610+0000 7fe360000640 1 Processor -- start 2024-08-17T02:46:21.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.610+0000 7fe360000640 1 -- start start 2024-08-17T02:46:21.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.611+0000 7fe360000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe358156d40 0x7fe35806dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:21.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.611+0000 7fe360000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe35806e420 0x7fe3580ab510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:21.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.611+0000 7fe360000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3581568b0 con 0x7fe358150130 2024-08-17T02:46:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.611+0000 7fe360000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35806e8a0 con 0x7fe35806e420 2024-08-17T02:46:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.611+0000 7fe360000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe358150500 con 0x7fe358156d40 2024-08-17T02:46:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.611+0000 7fe35e000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe35806e420 0x7fe3580ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.611+0000 7fe35e000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe35806e420 0x7fe3580ab510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43122/0 (socket says 172.21.15.16:43122) 2024-08-17T02:46:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.611+0000 7fe35e000640 1 -- 172.21.15.16:0/1288920016 learned_addr learned my addr 172.21.15.16:0/1288920016 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:21.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.611+0000 7fe35cc00640 1 --2- 172.21.15.16:0/1288920016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe358156d40 0x7fe35806dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.612+0000 7fe35e000640 1 -- 172.21.15.16:0/1288920016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe358156d40 msgr2=0x7fe35806dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.612+0000 7fe35e000640 1 --2- 172.21.15.16:0/1288920016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe358156d40 0x7fe35806dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.612+0000 7fe35e000640 1 -- 172.21.15.16:0/1288920016 >> v1:172.21.15.142:6789/0 conn(0x7fe358150130 legacy=0x7fe3580aa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.612+0000 7fe35e000640 1 -- 172.21.15.16:0/1288920016 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe358155fc0 con 0x7fe35806e420 2024-08-17T02:46:21.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.613+0000 7fe35e000640 1 --2- 172.21.15.16:0/1288920016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe35806e420 0x7fe3580ab510 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe350009f20 tx=0x7fe350009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e648ef656399a08b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.613+0000 7fe357e00640 1 -- 172.21.15.16:0/1288920016 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe350031400 con 0x7fe35806e420 2024-08-17T02:46:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.613+0000 7fe357e00640 1 -- 172.21.15.16:0/1288920016 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35002d050 con 0x7fe35806e420 2024-08-17T02:46:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.613+0000 7fe357e00640 1 -- 172.21.15.16:0/1288920016 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3500317d0 con 0x7fe35806e420 2024-08-17T02:46:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.614+0000 7fe360000640 1 -- 172.21.15.16:0/1288920016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe35806e420 msgr2=0x7fe3580ab510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.614+0000 7fe360000640 1 --2- 172.21.15.16:0/1288920016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe35806e420 0x7fe3580ab510 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe350009f20 tx=0x7fe350009840 comp rx=0 tx=0).stop 2024-08-17T02:46:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.614+0000 7fe360000640 1 -- 172.21.15.16:0/1288920016 shutdown_connections 2024-08-17T02:46:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.614+0000 7fe360000640 1 --2- 172.21.15.16:0/1288920016 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe35806e420 0x7fe3580ab510 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.614+0000 7fe360000640 1 --2- 172.21.15.16:0/1288920016 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe358156d40 0x7fe35806dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.614+0000 7fe360000640 1 -- 172.21.15.16:0/1288920016 >> 172.21.15.16:0/1288920016 conn(0x7fe358067750 msgr2=0x7fe35806c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.614+0000 7fe360000640 1 -- 172.21.15.16:0/1288920016 shutdown_connections 2024-08-17T02:46:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.614+0000 7fe360000640 1 -- 172.21.15.16:0/1288920016 wait complete. 2024-08-17T02:46:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.615+0000 7fe360000640 1 Processor -- start 2024-08-17T02:46:21.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.615+0000 7fe360000640 1 -- start start 2024-08-17T02:46:21.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.616+0000 7fe360000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35806e420 0x7fe358155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:21.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.616+0000 7fe360000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe358150130 0x7fe358158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:21.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.616+0000 7fe360000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe358156d40 0x7fe358168ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:21.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.616+0000 7fe360000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3581695b0 con 0x7fe358150130 2024-08-17T02:46:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.616+0000 7fe360000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe358169890 con 0x7fe35806e420 2024-08-17T02:46:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.616+0000 7fe360000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3581587e0 con 0x7fe358156d40 2024-08-17T02:46:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.616+0000 7fe35cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe358150130 0x7fe358158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.616+0000 7fe35cc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe358150130 0x7fe358158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43134/0 (socket says 172.21.15.16:43134) 2024-08-17T02:46:21.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.616+0000 7fe35cc00640 1 -- 172.21.15.16:0/1372730716 learned_addr learned my addr 172.21.15.16:0/1372730716 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:21.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe35d600640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35806e420 0x7fe358155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe35e000640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe358156d40 0x7fe358168ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe35cc00640 1 -- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe358156d40 msgr2=0x7fe358168ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe35cc00640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe358156d40 0x7fe358168ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe35cc00640 1 -- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35806e420 msgr2=0x7fe358155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe35cc00640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35806e420 0x7fe358155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe35cc00640 1 -- 172.21.15.16:0/1372730716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350009ae0 con 0x7fe358150130 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe35d600640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35806e420 0x7fe358155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe35e000640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe358156d40 0x7fe358168ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe35cc00640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe358150130 0x7fe358158fe0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe34400e8d0 tx=0x7fe3440090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe354c00640 1 -- 172.21.15.16:0/1372730716 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34400eec0 con 0x7fe358150130 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe354c00640 1 -- 172.21.15.16:0/1372730716 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34400c8e0 con 0x7fe358150130 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe360000640 1 -- 172.21.15.16:0/1372730716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe358158ac0 con 0x7fe358150130 2024-08-17T02:46:21.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe354c00640 1 -- 172.21.15.16:0/1372730716 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34400cc60 con 0x7fe358150130 2024-08-17T02:46:21.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.617+0000 7fe360000640 1 -- 172.21.15.16:0/1372730716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe358159b90 con 0x7fe358150130 2024-08-17T02:46:21.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.618+0000 7fe360000640 1 -- 172.21.15.16:0/1372730716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe358158640 con 0x7fe358150130 2024-08-17T02:46:21.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.619+0000 7fe354c00640 1 -- 172.21.15.16:0/1372730716 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fe344013680 con 0x7fe358150130 2024-08-17T02:46:21.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.620+0000 7fe354c00640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe338065150 0x7fe338067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:21.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.620+0000 7fe354c00640 1 -- 172.21.15.16:0/1372730716 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe3440825d0 con 0x7fe358150130 2024-08-17T02:46:21.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.621+0000 7fe35d600640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe338065150 0x7fe338067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:21.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.621+0000 7fe35d600640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe338065150 0x7fe338067600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0017a0 tx=0x7fe34c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:21.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.625+0000 7fe354c00640 1 -- 172.21.15.16:0/1372730716 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe358158640 con 0x7fe358150130 2024-08-17T02:46:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.892+0000 7fe360000640 1 -- 172.21.15.16:0/1372730716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe358158640 con 0x7fe358150130 2024-08-17T02:46:21.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.896+0000 7fe354c00640 1 -- 172.21.15.16:0/1372730716 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe358158640 con 0x7fe358150130 2024-08-17T02:46:21.896 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2024-08-17T02:46:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.898+0000 7fe360000640 1 -- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe338065150 msgr2=0x7fe338067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.898+0000 7fe360000640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe338065150 0x7fe338067600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0017a0 tx=0x7fe34c009040 comp rx=0 tx=0).stop 2024-08-17T02:46:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.898+0000 7fe360000640 1 -- 172.21.15.16:0/1372730716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe358150130 msgr2=0x7fe358158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.898+0000 7fe360000640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe358150130 0x7fe358158fe0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe34400e8d0 tx=0x7fe3440090f0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.898+0000 7fe360000640 1 -- 172.21.15.16:0/1372730716 shutdown_connections 2024-08-17T02:46:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.898+0000 7fe360000640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe338065150 0x7fe338067600 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.898+0000 7fe360000640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe358156d40 0x7fe358168ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.898+0000 7fe360000640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe358150130 0x7fe358158fe0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.898+0000 7fe360000640 1 --2- 172.21.15.16:0/1372730716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe35806e420 0x7fe358155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.898+0000 7fe360000640 1 -- 172.21.15.16:0/1372730716 >> 172.21.15.16:0/1372730716 conn(0x7fe358067750 msgr2=0x7fe3580aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.898+0000 7fe360000640 1 -- 172.21.15.16:0/1372730716 shutdown_connections 2024-08-17T02:46:21.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:21.898+0000 7fe360000640 1 -- 172.21.15.16:0/1372730716 wait complete. 2024-08-17T02:46:21.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:46:22.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.064+0000 7f11b2000640 1 Processor -- start 2024-08-17T02:46:22.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.065+0000 7f11b2000640 1 -- start start 2024-08-17T02:46:22.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.066+0000 7f11b2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0b0270 0x7f11ac1563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.066+0000 7f11b2000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f11ac158b30 0x7f11ac0ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.066+0000 7f11b2000640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ac157690 con 0x7f11ac155900 2024-08-17T02:46:22.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.066+0000 7f11b2000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ac156a80 con 0x7f11ac158b30 2024-08-17T02:46:22.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.066+0000 7f11b2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac156d60 con 0x7f11ac0b0270 2024-08-17T02:46:22.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.066+0000 7f11b0c00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f11ac155900 0x7f11ac155cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57300/0 (socket says 172.21.15.16:57300) 2024-08-17T02:46:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.066+0000 7f11b0c00640 1 -- 172.21.15.16:0/1584504360 learned_addr learned my addr 172.21.15.16:0/1584504360 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:22.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.066+0000 7f11b1600640 1 --2- 172.21.15.16:0/1584504360 >> v2:172.21.15.142:3300/0 conn(0x7f11ac158b30 0x7f11ac0ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.066+0000 7f11abe00640 1 --2- 172.21.15.16:0/1584504360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0b0270 0x7f11ac1563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.067+0000 7f11ab400640 1 -- 172.21.15.16:0/1584504360 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500930679 0 0) 0x7f11ac157690 con 0x7f11ac155900 2024-08-17T02:46:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.067+0000 7f11ab400640 1 -- 172.21.15.16:0/1584504360 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11940034e0 con 0x7f11ac155900 2024-08-17T02:46:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.067+0000 7f11abe00640 1 -- 172.21.15.16:0/1584504360 >> v2:172.21.15.142:3300/0 conn(0x7f11ac158b30 msgr2=0x7f11ac0ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.067+0000 7f11abe00640 1 --2- 172.21.15.16:0/1584504360 >> v2:172.21.15.142:3300/0 conn(0x7f11ac158b30 0x7f11ac0ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.067+0000 7f11abe00640 1 -- 172.21.15.16:0/1584504360 >> v1:172.21.15.16:6789/0 conn(0x7f11ac155900 legacy=0x7f11ac155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.067+0000 7f11abe00640 1 -- 172.21.15.16:0/1584504360 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac157040 con 0x7f11ac0b0270 2024-08-17T02:46:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.067+0000 7f11b1600640 1 --2- 172.21.15.16:0/1584504360 >> v2:172.21.15.142:3300/0 conn(0x7f11ac158b30 0x7f11ac0ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:46:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.068+0000 7f11abe00640 1 --2- 172.21.15.16:0/1584504360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0b0270 0x7f11ac1563e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1198009850 tx=0x7f11980090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f144b039e417f29 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:22.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.069+0000 7f11ab400640 1 -- 172.21.15.16:0/1584504360 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119802ad00 con 0x7f11ac0b0270 2024-08-17T02:46:22.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.069+0000 7f11ab400640 1 -- 172.21.15.16:0/1584504360 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119802ae60 con 0x7f11ac0b0270 2024-08-17T02:46:22.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.069+0000 7f11ab400640 1 -- 172.21.15.16:0/1584504360 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1198032670 con 0x7f11ac0b0270 2024-08-17T02:46:22.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.070+0000 7f11b2000640 1 -- 172.21.15.16:0/1584504360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0b0270 msgr2=0x7f11ac1563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.070+0000 7f11b2000640 1 --2- 172.21.15.16:0/1584504360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0b0270 0x7f11ac1563e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1198009850 tx=0x7f11980090f0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.070+0000 7f11b2000640 1 -- 172.21.15.16:0/1584504360 shutdown_connections 2024-08-17T02:46:22.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.070+0000 7f11b2000640 1 --2- 172.21.15.16:0/1584504360 >> v2:172.21.15.142:3300/0 conn(0x7f11ac158b30 0x7f11ac0ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.070+0000 7f11b2000640 1 --2- 172.21.15.16:0/1584504360 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0b0270 0x7f11ac1563e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.070+0000 7f11b2000640 1 -- 172.21.15.16:0/1584504360 >> 172.21.15.16:0/1584504360 conn(0x7f11ac067750 msgr2=0x7f11ac0cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:22.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.070+0000 7f11b2000640 1 -- 172.21.15.16:0/1584504360 shutdown_connections 2024-08-17T02:46:22.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.070+0000 7f11b2000640 1 -- 172.21.15.16:0/1584504360 wait complete. 2024-08-17T02:46:22.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.071+0000 7f11b2000640 1 Processor -- start 2024-08-17T02:46:22.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.071+0000 7f11b2000640 1 -- start start 2024-08-17T02:46:22.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.072+0000 7f11b2000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0b0270 0x7f11ac150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.072+0000 7f11b2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac155900 0x7f11ac0a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.072+0000 7f11b2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac158b30 0x7f11ac0c11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.072+0000 7f11b2000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac0c2d40 con 0x7f11ac158b30 2024-08-17T02:46:22.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.072+0000 7f11b2000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac0a9aa0 con 0x7f11ac155900 2024-08-17T02:46:22.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.072+0000 7f11b2000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac0a8e90 con 0x7f11ac0b0270 2024-08-17T02:46:22.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.073+0000 7f11b1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac158b30 0x7f11ac0c11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.073+0000 7f11b1600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac158b30 0x7f11ac0c11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43148/0 (socket says 172.21.15.16:43148) 2024-08-17T02:46:22.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.073+0000 7f11b1600640 1 -- 172.21.15.16:0/2508255889 learned_addr learned my addr 172.21.15.16:0/2508255889 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:22.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.073+0000 7f11b0c00640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0b0270 0x7f11ac150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.073+0000 7f11abe00640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac155900 0x7f11ac0a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.073+0000 7f11b1600640 1 -- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0b0270 msgr2=0x7f11ac150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.073+0000 7f11b1600640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0b0270 0x7f11ac150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.073+0000 7f11b1600640 1 -- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac155900 msgr2=0x7f11ac0a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.073+0000 7f11b1600640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac155900 0x7f11ac0a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.073+0000 7f11b1600640 1 -- 172.21.15.16:0/2508255889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a4009a20 con 0x7f11ac158b30 2024-08-17T02:46:22.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.073+0000 7f11b0c00640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0b0270 0x7f11ac150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:22.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.073+0000 7f11b1600640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac158b30 0x7f11ac0c11e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009f20 tx=0x7f11a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:22.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.074+0000 7f119fe00640 1 -- 172.21.15.16:0/2508255889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a400ce10 con 0x7f11ac158b30 2024-08-17T02:46:22.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.074+0000 7f119fe00640 1 -- 172.21.15.16:0/2508255889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a400f040 con 0x7f11ac158b30 2024-08-17T02:46:22.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.074+0000 7f119fe00640 1 -- 172.21.15.16:0/2508255889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a40146d0 con 0x7f11ac158b30 2024-08-17T02:46:22.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.074+0000 7f11b2000640 1 -- 172.21.15.16:0/2508255889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1198009390 con 0x7f11ac158b30 2024-08-17T02:46:22.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.074+0000 7f11b2000640 1 -- 172.21.15.16:0/2508255889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ac0a94d0 con 0x7f11ac158b30 2024-08-17T02:46:22.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.075+0000 7f11b2000640 1 -- 172.21.15.16:0/2508255889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ac071050 con 0x7f11ac158b30 2024-08-17T02:46:22.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.076+0000 7f119fe00640 1 -- 172.21.15.16:0/2508255889 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f11a4003020 con 0x7f11ac158b30 2024-08-17T02:46:22.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.076+0000 7f119fe00640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f118c065150 0x7f118c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.077+0000 7f119fe00640 1 -- 172.21.15.16:0/2508255889 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f11a4082ec0 con 0x7f11ac158b30 2024-08-17T02:46:22.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.077+0000 7f11b0c00640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f118c065150 0x7f118c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.078+0000 7f11b0c00640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f118c065150 0x7f118c067600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f11ac152dc0 tx=0x7f11a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:22.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.080+0000 7f119fe00640 1 -- 172.21.15.16:0/2508255889 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f11a4050f80 con 0x7f11ac158b30 2024-08-17T02:46:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.300+0000 7f11b2000640 1 -- 172.21.15.16:0/2508255889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11ac0a97f0 con 0x7f11ac158b30 2024-08-17T02:46:22.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.300+0000 7f119fe00640 1 -- 172.21.15.16:0/2508255889 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f11a40545a0 con 0x7f11ac158b30 2024-08-17T02:46:22.301 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:46:22.301 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2889208700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2889208700},{"type":"v1","addr":"172.21.15.142:6837","nonce":2889208700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4287834697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4287834697},{"type":"v1","addr":"172.21.15.142:6835","nonce":4287834697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4953,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3115083427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3115083427},{"type":"v1","addr":"172.21.15.16:6834","nonce":3115083427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:16.190199+0000","modified":"2024-08-17T02:46:19.780153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6835/219684854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":219684854},{"type":"v1","addr":"172.21.15.16:6835","nonce":219684854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2024-08-17T02:46:22.301 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 18 2024-08-17T02:46:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.303+0000 7f11b2000640 1 -- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f118c065150 msgr2=0x7f118c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.303+0000 7f11b2000640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f118c065150 0x7f118c067600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f11ac152dc0 tx=0x7f11a0009040 comp rx=0 tx=0).stop 2024-08-17T02:46:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.303+0000 7f11b2000640 1 -- 172.21.15.16:0/2508255889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac158b30 msgr2=0x7f11ac0c11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.303+0000 7f11b2000640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac158b30 0x7f11ac0c11e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009f20 tx=0x7f11a400b040 comp rx=0 tx=0).stop 2024-08-17T02:46:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.303+0000 7f11b2000640 1 -- 172.21.15.16:0/2508255889 shutdown_connections 2024-08-17T02:46:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.303+0000 7f11b2000640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f118c065150 0x7f118c067600 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.303+0000 7f11b2000640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11ac158b30 0x7f11ac0c11e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.303+0000 7f11b2000640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac155900 0x7f11ac0a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.303+0000 7f11b2000640 1 --2- 172.21.15.16:0/2508255889 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f11ac0b0270 0x7f11ac150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.303+0000 7f11b2000640 1 -- 172.21.15.16:0/2508255889 >> 172.21.15.16:0/2508255889 conn(0x7f11ac067750 msgr2=0x7f11ac151e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.303+0000 7f11b2000640 1 -- 172.21.15.16:0/2508255889 shutdown_connections 2024-08-17T02:46:22.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.303+0000 7f11b2000640 1 -- 172.21.15.16:0/2508255889 wait complete. 2024-08-17T02:46:22.720 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-17T02:46:16.190199+0000', 'modified': '2024-08-17T02:46:19.780153+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4956}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4956': {'gid': 4956, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.16:6835/219684854', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.16:6833', 'nonce': 219684854}, {'type': 'v1', 'addr': '172.21.15.16:6835', 'nonce': 219684854}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2024-08-17T02:46:22.720 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-17T02:46:22.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-08-17T02:46:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.874+0000 7f77a4400640 1 Processor -- start 2024-08-17T02:46:22.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.874+0000 7f77a4400640 1 -- start start 2024-08-17T02:46:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.875+0000 7f77a4400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f779c13f4a0 0x7f779c0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.875+0000 7f77a4400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f779c06bf30 0x7f779c0ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.875+0000 7f77a4400640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779c06baa0 con 0x7f779c06a8d0 2024-08-17T02:46:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.875+0000 7f77a4400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779c06ae90 con 0x7f779c06bf30 2024-08-17T02:46:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.875+0000 7f77a4400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779c06b170 con 0x7f779c13f4a0 2024-08-17T02:46:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.875+0000 7f77a1000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f779c06bf30 0x7f779c0ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.875+0000 7f77a1000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f779c06bf30 0x7f779c0ae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43150/0 (socket says 172.21.15.16:43150) 2024-08-17T02:46:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.875+0000 7f77a1000640 1 -- 172.21.15.16:0/1266555425 learned_addr learned my addr 172.21.15.16:0/1266555425 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:22.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.875+0000 7f77a2400640 1 --1- 172.21.15.16:0/1266555425 >> v1:172.21.15.16:6789/0 conn(0x7f779c06a8d0 0x7f779c0ca210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57302/0 (socket says 172.21.15.16:57302) 2024-08-17T02:46:22.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.876+0000 7f77a1a00640 1 --2- 172.21.15.16:0/1266555425 >> v2:172.21.15.142:3300/0 conn(0x7f779c13f4a0 0x7f779c0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.876+0000 7f77a1000640 1 -- 172.21.15.16:0/1266555425 >> v2:172.21.15.142:3300/0 conn(0x7f779c13f4a0 msgr2=0x7f779c0ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:22.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.876+0000 7f77a1000640 1 --2- 172.21.15.16:0/1266555425 >> v2:172.21.15.142:3300/0 conn(0x7f779c13f4a0 0x7f779c0ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.876+0000 7f77a1000640 1 -- 172.21.15.16:0/1266555425 >> v1:172.21.15.16:6789/0 conn(0x7f779c06a8d0 legacy=0x7f779c0ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.876+0000 7f77a1000640 1 -- 172.21.15.16:0/1266555425 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c06b450 con 0x7f779c06bf30 2024-08-17T02:46:22.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.876+0000 7f77a1a00640 1 --2- 172.21.15.16:0/1266555425 >> v2:172.21.15.142:3300/0 conn(0x7f779c13f4a0 0x7f779c0ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:22.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.877+0000 7f77a1000640 1 --2- 172.21.15.16:0/1266555425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c06bf30 0x7f779c0ae470 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f779c06d6f0 tx=0x7f77880091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49eb531bc98e6ade server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:22.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.877+0000 7f779be00640 1 -- 172.21.15.16:0/1266555425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778802add0 con 0x7f779c06bf30 2024-08-17T02:46:22.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.877+0000 7f779be00640 1 -- 172.21.15.16:0/1266555425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7788032440 con 0x7f779c06bf30 2024-08-17T02:46:22.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.877+0000 7f779be00640 1 -- 172.21.15.16:0/1266555425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788032820 con 0x7f779c06bf30 2024-08-17T02:46:22.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.878+0000 7f77a4400640 1 -- 172.21.15.16:0/1266555425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c06bf30 msgr2=0x7f779c0ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.878+0000 7f77a4400640 1 --2- 172.21.15.16:0/1266555425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c06bf30 0x7f779c0ae470 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f779c06d6f0 tx=0x7f77880091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.878+0000 7f77a4400640 1 -- 172.21.15.16:0/1266555425 shutdown_connections 2024-08-17T02:46:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.878+0000 7f77a4400640 1 --2- 172.21.15.16:0/1266555425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c06bf30 0x7f779c0ae470 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.878+0000 7f77a4400640 1 --2- 172.21.15.16:0/1266555425 >> v2:172.21.15.142:3300/0 conn(0x7f779c13f4a0 0x7f779c0ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.878+0000 7f77a4400640 1 -- 172.21.15.16:0/1266555425 >> 172.21.15.16:0/1266555425 conn(0x7f779c06d8f0 msgr2=0x7f779c06dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.878+0000 7f77a4400640 1 -- 172.21.15.16:0/1266555425 shutdown_connections 2024-08-17T02:46:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.878+0000 7f77a4400640 1 -- 172.21.15.16:0/1266555425 wait complete. 2024-08-17T02:46:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.879+0000 7f77a4400640 1 Processor -- start 2024-08-17T02:46:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.879+0000 7f77a4400640 1 -- start start 2024-08-17T02:46:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.880+0000 7f77a4400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c06a8d0 0x7f779c155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.880+0000 7f77a4400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f779c06bf30 0x7f779c0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.880+0000 7f77a4400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f779c13f4a0 0x7f779c0c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.880+0000 7f77a4400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c169b90 con 0x7f779c06a8d0 2024-08-17T02:46:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.880+0000 7f77a4400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c13dc90 con 0x7f779c06bf30 2024-08-17T02:46:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.880+0000 7f77a4400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779c0c2280 con 0x7f779c13f4a0 2024-08-17T02:46:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.880+0000 7f77a1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c06a8d0 0x7f779c155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.881+0000 7f77a1a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c06a8d0 0x7f779c155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43164/0 (socket says 172.21.15.16:43164) 2024-08-17T02:46:22.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.881+0000 7f77a1a00640 1 -- 172.21.15.16:0/887027163 learned_addr learned my addr 172.21.15.16:0/887027163 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.881+0000 7f77a1000640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f779c06bf30 0x7f779c0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.881+0000 7f77a2400640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f779c13f4a0 0x7f779c0c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.881+0000 7f77a1a00640 1 -- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f779c13f4a0 msgr2=0x7f779c0c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.881+0000 7f77a1a00640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f779c13f4a0 0x7f779c0c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.881+0000 7f77a1a00640 1 -- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f779c06bf30 msgr2=0x7f779c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.881+0000 7f77a1a00640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f779c06bf30 0x7f779c0c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.881+0000 7f77a1a00640 1 -- 172.21.15.16:0/887027163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7788009450 con 0x7f779c06a8d0 2024-08-17T02:46:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.881+0000 7f77a1a00640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c06a8d0 0x7f779c155280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f779000d810 tx=0x7f7790008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.882+0000 7f7798c00640 1 -- 172.21.15.16:0/887027163 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779000de00 con 0x7f779c06a8d0 2024-08-17T02:46:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.882+0000 7f7798c00640 1 -- 172.21.15.16:0/887027163 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77900098b0 con 0x7f779c06a8d0 2024-08-17T02:46:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.882+0000 7f7798c00640 1 -- 172.21.15.16:0/887027163 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790009c70 con 0x7f779c06a8d0 2024-08-17T02:46:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.882+0000 7f77a4400640 1 -- 172.21.15.16:0/887027163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779c1557c0 con 0x7f779c06a8d0 2024-08-17T02:46:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.882+0000 7f77a4400640 1 -- 172.21.15.16:0/887027163 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779c0c1c30 con 0x7f779c06a8d0 2024-08-17T02:46:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.884+0000 7f7798c00640 1 -- 172.21.15.16:0/887027163 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f7790014640 con 0x7f779c06a8d0 2024-08-17T02:46:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.884+0000 7f7798c00640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f777c065150 0x7f777c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.884+0000 7f7798c00640 1 -- 172.21.15.16:0/887027163 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7790082890 con 0x7f779c06a8d0 2024-08-17T02:46:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.885+0000 7f77a1000640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f777c065150 0x7f777c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.885+0000 7f77a687f740 1 Processor -- start 2024-08-17T02:46:22.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.885+0000 7f77a687f740 1 -- start start 2024-08-17T02:46:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.886+0000 7f77a1000640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f777c065150 0x7f777c067600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7788034000 tx=0x7f7788032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.886+0000 7f77a687f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2efca30 0x5643b2efce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.886+0000 7f77a687f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2ed80a0 0x5643b2f1a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.886+0000 7f77a687f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2f1ab10 0x5643b2ed1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.886+0000 7f77a687f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b2e92380 con 0x5643b2f1ab10 2024-08-17T02:46:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.886+0000 7f77a687f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b2e3b9e0 con 0x5643b2efca30 2024-08-17T02:46:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.886+0000 7f77a687f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643b2e39170 con 0x5643b2ed80a0 2024-08-17T02:46:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.886+0000 7f77a2400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2f1ab10 0x5643b2ed1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.886+0000 7f77a1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2efca30 0x5643b2efce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.886+0000 7f77a1000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2ed80a0 0x5643b2f1a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.886+0000 7f77a1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2efca30 0x5643b2efce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:53740/0 (socket says 172.21.15.16:53740) 2024-08-17T02:46:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.886+0000 7f77a1a00640 1 -- 172.21.15.16:0/1381912083 learned_addr learned my addr 172.21.15.16:0/1381912083 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.887+0000 7f77a2400640 1 -- 172.21.15.16:0/1381912083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2ed80a0 msgr2=0x5643b2f1a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.887+0000 7f77a2400640 1 --2- 172.21.15.16:0/1381912083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2ed80a0 0x5643b2f1a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.887+0000 7f77a2400640 1 -- 172.21.15.16:0/1381912083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2efca30 msgr2=0x5643b2efce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.887+0000 7f77a2400640 1 --2- 172.21.15.16:0/1381912083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2efca30 0x5643b2efce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.887+0000 7f77a2400640 1 -- 172.21.15.16:0/1381912083 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643b2de8820 con 0x5643b2f1ab10 2024-08-17T02:46:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.887+0000 7f77a2400640 1 --2- 172.21.15.16:0/1381912083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2f1ab10 0x5643b2ed1900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f779c0c2230 tx=0x7f7794005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cdfd420024541f server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.887+0000 7f778d600640 1 -- 172.21.15.16:0/1381912083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794007a00 con 0x5643b2f1ab10 2024-08-17T02:46:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.887+0000 7f778d600640 1 -- 172.21.15.16:0/1381912083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794001c90 con 0x5643b2f1ab10 2024-08-17T02:46:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.887+0000 7f778d600640 1 -- 172.21.15.16:0/1381912083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794002010 con 0x5643b2f1ab10 2024-08-17T02:46:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.888+0000 7f77a687f740 1 -- 172.21.15.16:0/1381912083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2f1ab10 msgr2=0x5643b2ed1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.888+0000 7f77a687f740 1 --2- 172.21.15.16:0/1381912083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2f1ab10 0x5643b2ed1900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f779c0c2230 tx=0x7f7794005e10 comp rx=0 tx=0).stop 2024-08-17T02:46:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.888+0000 7f77a687f740 1 -- 172.21.15.16:0/1381912083 shutdown_connections 2024-08-17T02:46:22.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.888+0000 7f77a687f740 1 --2- 172.21.15.16:0/1381912083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2f1ab10 0x5643b2ed1900 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.888+0000 7f77a687f740 1 --2- 172.21.15.16:0/1381912083 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2ed80a0 0x5643b2f1a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.888+0000 7f77a687f740 1 --2- 172.21.15.16:0/1381912083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2efca30 0x5643b2efce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.888+0000 7f77a687f740 1 -- 172.21.15.16:0/1381912083 >> 172.21.15.16:0/1381912083 conn(0x5643b2e70030 msgr2=0x5643b2f1ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.888+0000 7f77a687f740 1 -- 172.21.15.16:0/1381912083 shutdown_connections 2024-08-17T02:46:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.888+0000 7f77a687f740 1 -- 172.21.15.16:0/1381912083 wait complete. 2024-08-17T02:46:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.889+0000 7f77a687f740 1 -- start start 2024-08-17T02:46:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.889+0000 7f77a687f740 1 Processor -- start 2024-08-17T02:46:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.890+0000 7f77a687f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2ed80a0 0x5643b2f03270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.890+0000 7f77a687f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2efadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.890+0000 7f77a687f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2f1ab10 0x5643b2ef91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.890+0000 7f77a687f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b2e34f60 con 0x5643b2efca30 2024-08-17T02:46:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.890+0000 7f77a687f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b2ddd690 con 0x5643b2f1ab10 2024-08-17T02:46:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.890+0000 7f77a687f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643b2ddd330 con 0x5643b2ed80a0 2024-08-17T02:46:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.890+0000 7f77a687f740 10 client.0 _ll_register_callbacks cb 0x5643b2dd72b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-17T02:46:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.890+0000 7f77a1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2efadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.890+0000 7f77a1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2efadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43176/0 (socket says 172.21.15.16:43176) 2024-08-17T02:46:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.890+0000 7f77a1000640 1 -- 172.21.15.16:0/2500468288 learned_addr learned my addr 172.21.15.16:0/2500468288 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.890+0000 7f77a1a00640 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2ed80a0 0x5643b2f03270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.890+0000 7f77a2400640 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2f1ab10 0x5643b2ef91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f77a1000640 1 -- 172.21.15.16:0/2500468288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2ed80a0 msgr2=0x5643b2f03270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f77a1000640 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2ed80a0 0x5643b2f03270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f77a1000640 1 -- 172.21.15.16:0/2500468288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2f1ab10 msgr2=0x5643b2ef91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f77a1000640 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2f1ab10 0x5643b2ef91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f77a1000640 1 -- 172.21.15.16:0/2500468288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779400f040 con 0x5643b2efca30 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f77a1000640 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2efadf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7788009420 tx=0x7f778803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d10371912e30ddf server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f7782000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f7782000640 1 -- 172.21.15.16:0/2500468288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778803ce70 con 0x5643b2efca30 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f7782000640 1 -- 172.21.15.16:0/2500468288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7788042040 con 0x5643b2efca30 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f77a687f740 1 -- 172.21.15.16:0/2500468288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643b2e68d20 con 0x5643b2efca30 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f7782000640 1 -- 172.21.15.16:0/2500468288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7788031b00 con 0x5643b2efca30 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f77a687f740 10 client.5145 fetch_fsmap learned FSMap version 18 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.891+0000 7f77a687f740 1 -- 172.21.15.16:0/2500468288 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5643b2dfeab0 con 0x5643b2efca30 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.892+0000 7f7782000640 1 -- 172.21.15.16:0/2500468288 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5643b2e68d20 con 0x5643b2efca30 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.892+0000 7f7782000640 1 -- 172.21.15.16:0/2500468288 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 18) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7788031c60 con 0x5643b2efca30 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.892+0000 7f77a687f740 10 client.5145 fetch_fsmap finished waiting for FSMap version 18 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.892+0000 7f77a687f740 10 client.5145 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-17T02:46:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.892+0000 7f77a687f740 20 client.5145 populate_metadata read hostname 'smithi016' 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.892+0000 7f77a687f740 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.16:6833/219684854,v1:172.21.15.16:6835/219684854] conn(0x5643b2f2acb0 0x5643b2f01e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.892+0000 7f77a687f740 4 client.5145 mds_command: new command op to 4956 tid=0[{"prefix": "get_command_descriptions"}] 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.892+0000 7f77a687f740 1 -- 172.21.15.16:0/2500468288 --> [v2:172.21.15.16:6833/219684854,v1:172.21.15.16:6835/219684854] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643b2b3c3c0 con 0x5643b2f2acb0 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.892+0000 7f77a1a00640 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.16:6833/219684854,v1:172.21.15.16:6835/219684854] conn(0x5643b2f2acb0 0x5643b2f01e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.893+0000 7f77a1a00640 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.16:6833/219684854,v1:172.21.15.16:6835/219684854] conn(0x5643b2f2acb0 0x5643b2f01e70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=823698c793fb21d server_cookie=b7b156d8d8b6ef57 in_seq=0 out_seq=0 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.893+0000 7f7782000640 10 client.5145 ms_handle_connect on v2:172.21.15.16:6833/219684854 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.893+0000 7f7782000640 1 -- 172.21.15.16:0/2500468288 <== mds.0 v2:172.21.15.16:6833/219684854 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17536 (crc 0 0 0) 0x5643b2b3c3c0 con 0x5643b2f2acb0 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.893+0000 7f7782000640 10 client.5145 handle_command_reply: tid=0 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.893+0000 7f77a687f740 1 client.5145 shutdown 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.894+0000 7f77a687f740 1 -- 172.21.15.16:0/2500468288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 msgr2=0x5643b2efadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.894+0000 7f77a687f740 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2efadf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7788009420 tx=0x7f778803c000 comp rx=0 tx=0).stop 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.894+0000 7f77a687f740 1 -- 172.21.15.16:0/2500468288 shutdown_connections 2024-08-17T02:46:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.894+0000 7f77a687f740 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.16:6833/219684854,v1:172.21.15.16:6835/219684854] conn(0x5643b2f2acb0 0x5643b2f01e70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.894+0000 7f77a687f740 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2f1ab10 0x5643b2ef91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.894+0000 7f77a687f740 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2efadf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.894+0000 7f77a687f740 1 --2- 172.21.15.16:0/2500468288 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2ed80a0 0x5643b2f03270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.894+0000 7f77a687f740 1 -- 172.21.15.16:0/2500468288 >> 172.21.15.16:0/2500468288 conn(0x5643b2e70030 msgr2=0x5643b2f001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.894+0000 7f7782000640 0 client.5145 ms_handle_reset on v2:172.21.15.16:6833/219684854 2024-08-17T02:46:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.894+0000 7f77a687f740 1 -- 172.21.15.16:0/2500468288 shutdown_connections 2024-08-17T02:46:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.894+0000 7f77a687f740 1 -- 172.21.15.16:0/2500468288 wait complete. 2024-08-17T02:46:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.894+0000 7f77a687f740 20 client.5145 trim_cache size 0 max 16384 2024-08-17T02:46:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.911+0000 7f77a687f740 1 Processor -- start 2024-08-17T02:46:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.911+0000 7f77a687f740 1 -- start start 2024-08-17T02:46:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.912+0000 7f77a687f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2ed80a0 0x5643b2f1f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.912+0000 7f77a687f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2f01ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.912+0000 7f77a687f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2f1ab10 0x5643b2f02b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.912+0000 7f77a687f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b2baff20 con 0x5643b2efca30 2024-08-17T02:46:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.912+0000 7f77a687f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b2ed74e0 con 0x5643b2ed80a0 2024-08-17T02:46:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.912+0000 7f77a687f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643b2f04b80 con 0x5643b2f1ab10 2024-08-17T02:46:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.913+0000 7f77a1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2f01ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.913+0000 7f77a1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2f01ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43178/0 (socket says 172.21.15.16:43178) 2024-08-17T02:46:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.913+0000 7f77a1000640 1 -- 172.21.15.16:0/2321525585 learned_addr learned my addr 172.21.15.16:0/2321525585 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.913+0000 7f77a1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2ed80a0 0x5643b2f1f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.913+0000 7f77a2400640 1 --2- 172.21.15.16:0/2321525585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2f1ab10 0x5643b2f02b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.913+0000 7f77a1000640 1 -- 172.21.15.16:0/2321525585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2f1ab10 msgr2=0x5643b2f02b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.913+0000 7f77a1000640 1 --2- 172.21.15.16:0/2321525585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2f1ab10 0x5643b2f02b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.913+0000 7f77a1000640 1 -- 172.21.15.16:0/2321525585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2ed80a0 msgr2=0x5643b2f1f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.913+0000 7f77a1000640 1 --2- 172.21.15.16:0/2321525585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2ed80a0 0x5643b2f1f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.913+0000 7f77a1000640 1 -- 172.21.15.16:0/2321525585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643b2e02490 con 0x5643b2efca30 2024-08-17T02:46:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.913+0000 7f77a1000640 1 --2- 172.21.15.16:0/2321525585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2f01ed0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f77880326b0 tx=0x7f778800b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9319b79c2e664986 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:22.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.913+0000 7f778e000640 1 -- 172.21.15.16:0/2321525585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788032980 con 0x5643b2efca30 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.914+0000 7f778e000640 1 -- 172.21.15.16:0/2321525585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7788032ae0 con 0x5643b2efca30 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.914+0000 7f778e000640 1 -- 172.21.15.16:0/2321525585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788031d10 con 0x5643b2efca30 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.914+0000 7f77a687f740 1 -- 172.21.15.16:0/2321525585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 msgr2=0x5643b2f01ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.914+0000 7f77a687f740 1 --2- 172.21.15.16:0/2321525585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2f01ed0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f77880326b0 tx=0x7f778800b050 comp rx=0 tx=0).stop 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.914+0000 7f77a687f740 1 -- 172.21.15.16:0/2321525585 shutdown_connections 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.914+0000 7f77a687f740 1 --2- 172.21.15.16:0/2321525585 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2f1ab10 0x5643b2f02b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.914+0000 7f77a687f740 1 --2- 172.21.15.16:0/2321525585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2f01ed0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.914+0000 7f77a687f740 1 --2- 172.21.15.16:0/2321525585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2ed80a0 0x5643b2f1f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.914+0000 7f77a687f740 1 -- 172.21.15.16:0/2321525585 >> 172.21.15.16:0/2321525585 conn(0x5643b2e70030 msgr2=0x5643b2ecc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.914+0000 7f77a687f740 1 -- 172.21.15.16:0/2321525585 shutdown_connections 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.914+0000 7f77a687f740 1 -- 172.21.15.16:0/2321525585 wait complete. 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.915+0000 7f77a687f740 1 -- start start 2024-08-17T02:46:22.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.915+0000 7f77a687f740 1 Processor -- start 2024-08-17T02:46:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.916+0000 7f77a687f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2ed80a0 0x5643b2f396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.916+0000 7f77a687f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2f81350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.916+0000 7f77a687f740 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2f1ab10 0x5643b2f81890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.916+0000 7f77a687f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b2f36f90 con 0x5643b2efca30 2024-08-17T02:46:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.916+0000 7f77a687f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643b2f36940 con 0x5643b2ed80a0 2024-08-17T02:46:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.916+0000 7f77a687f740 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643b2f36c20 con 0x5643b2f1ab10 2024-08-17T02:46:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.916+0000 7f77a687f740 10 client.0 _ll_register_callbacks cb 0x7f778803c890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-17T02:46:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.916+0000 7f77a1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2f81350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.916+0000 7f77a1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2f81350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43194/0 (socket says 172.21.15.16:43194) 2024-08-17T02:46:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.916+0000 7f77a1000640 1 -- 172.21.15.16:0/796978500 learned_addr learned my addr 172.21.15.16:0/796978500 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.916+0000 7f77a1a00640 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2ed80a0 0x5643b2f396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.917+0000 7f77a2400640 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2f1ab10 0x5643b2f81890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.917+0000 7f77a1000640 1 -- 172.21.15.16:0/796978500 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2f1ab10 msgr2=0x5643b2f81890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.917+0000 7f77a1000640 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2f1ab10 0x5643b2f81890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.917+0000 7f77a1000640 1 -- 172.21.15.16:0/796978500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2ed80a0 msgr2=0x5643b2f396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.917+0000 7f77a1000640 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2ed80a0 0x5643b2f396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.917+0000 7f77a1000640 1 -- 172.21.15.16:0/796978500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643b2dfeab0 con 0x5643b2efca30 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.917+0000 7f77a1000640 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2f81350 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f778800b0b0 tx=0x7f7788049040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f267c8622d8ddd1c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.917+0000 7f7782000640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.917+0000 7f7782000640 1 -- 172.21.15.16:0/796978500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788031de0 con 0x5643b2efca30 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.917+0000 7f7782000640 1 -- 172.21.15.16:0/796978500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7788042040 con 0x5643b2efca30 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.917+0000 7f77a687f740 1 -- 172.21.15.16:0/796978500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7794001c90 con 0x5643b2efca30 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.918+0000 7f7782000640 1 -- 172.21.15.16:0/796978500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7788050790 con 0x5643b2efca30 2024-08-17T02:46:22.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.918+0000 7f77a687f740 10 client.5151 fetch_fsmap learned FSMap version 18 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.918+0000 7f77a687f740 1 -- 172.21.15.16:0/796978500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x5643b2e021e0 con 0x5643b2efca30 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.918+0000 7f7782000640 1 -- 172.21.15.16:0/796978500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794001c90 con 0x5643b2efca30 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.918+0000 7f7782000640 1 -- 172.21.15.16:0/796978500 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 18) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f778805b460 con 0x5643b2efca30 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.918+0000 7f77a687f740 10 client.5151 fetch_fsmap finished waiting for FSMap version 18 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.918+0000 7f77a687f740 10 client.5151 resolve_mds: resolved name 'a' to daemon mds.a 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.918+0000 7f77a687f740 20 client.5151 populate_metadata read hostname 'smithi016' 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.918+0000 7f77a687f740 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.16:6833/219684854,v1:172.21.15.16:6835/219684854] conn(0x5643b2f37420 0x5643b2f377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.918+0000 7f77a687f740 4 client.5151 mds_command: new command op to 4956 tid=0[{"prefix": "status"}] 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.918+0000 7f77a687f740 1 -- 172.21.15.16:0/796978500 --> [v2:172.21.15.16:6833/219684854,v1:172.21.15.16:6835/219684854] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5643b2b4d6c0 con 0x5643b2f37420 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.918+0000 7f77a1a00640 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.16:6833/219684854,v1:172.21.15.16:6835/219684854] conn(0x5643b2f37420 0x5643b2f377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f77a1a00640 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.16:6833/219684854,v1:172.21.15.16:6835/219684854] conn(0x5643b2f37420 0x5643b2f377f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edef61e2c64cf92b server_cookie=9e76b60e8d7cbb6d in_seq=0 out_seq=0 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f7782000640 10 client.5151 ms_handle_connect on v2:172.21.15.16:6833/219684854 2024-08-17T02:46:22.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f7782000640 1 -- 172.21.15.16:0/796978500 <== mds.0 v2:172.21.15.16:6833/219684854 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5643b2b4d6c0 con 0x5643b2f37420 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f7782000640 10 client.5151 handle_command_reply: tid=0 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f77a687f740 1 client.5151 shutdown 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f77a687f740 1 -- 172.21.15.16:0/796978500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 msgr2=0x5643b2f81350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f77a687f740 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2f81350 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f778800b0b0 tx=0x7f7788049040 comp rx=0 tx=0).stop 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f77a687f740 1 -- 172.21.15.16:0/796978500 shutdown_connections 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f77a687f740 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.16:6833/219684854,v1:172.21.15.16:6835/219684854] conn(0x5643b2f37420 0x5643b2f377f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f77a687f740 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x5643b2f1ab10 0x5643b2f81890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f77a687f740 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5643b2efca30 0x5643b2f81350 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f77a687f740 1 --2- 172.21.15.16:0/796978500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5643b2ed80a0 0x5643b2f396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f7782000640 0 client.5151 ms_handle_reset on v2:172.21.15.16:6833/219684854 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f77a687f740 1 -- 172.21.15.16:0/796978500 >> 172.21.15.16:0/796978500 conn(0x5643b2e70030 msgr2=0x5643b2ecc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f77a687f740 1 -- 172.21.15.16:0/796978500 shutdown_connections 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.919+0000 7f77a687f740 1 -- 172.21.15.16:0/796978500 wait complete. 2024-08-17T02:46:22.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a687f740 20 client.5151 trim_cache size 0 max 16384 2024-08-17T02:46:22.925 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-17T02:46:22.925 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778", 2024-08-17T02:46:22.925 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-08-17T02:46:22.925 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4956, 2024-08-17T02:46:22.925 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-08-17T02:46:22.925 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-08-17T02:46:22.925 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-08-17T02:46:22.925 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 6.1507666370000003, 2024-08-17T02:46:22.925 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 18, 2024-08-17T02:46:22.926 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 33, 2024-08-17T02:46:22.926 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 33, 2024-08-17T02:46:22.926 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 20.986131792999998 2024-08-17T02:46:22.926 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-17T02:46:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a4400640 1 -- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f777c065150 msgr2=0x7f777c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a4400640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f777c065150 0x7f777c067600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7788034000 tx=0x7f7788032000 comp rx=0 tx=0).stop 2024-08-17T02:46:22.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a4400640 1 -- 172.21.15.16:0/887027163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c06a8d0 msgr2=0x7f779c155280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:22.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a4400640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c06a8d0 0x7f779c155280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f779000d810 tx=0x7f7790008040 comp rx=0 tx=0).stop 2024-08-17T02:46:22.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a4400640 1 -- 172.21.15.16:0/887027163 shutdown_connections 2024-08-17T02:46:22.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a4400640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f777c065150 0x7f777c067600 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a4400640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f779c13f4a0 0x7f779c0c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a4400640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f779c06bf30 0x7f779c0c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a4400640 1 --2- 172.21.15.16:0/887027163 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779c06a8d0 0x7f779c155280 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:22.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a4400640 1 -- 172.21.15.16:0/887027163 >> 172.21.15.16:0/887027163 conn(0x7f779c06d8f0 msgr2=0x7f779c0670a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:22.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a4400640 1 -- 172.21.15.16:0/887027163 shutdown_connections 2024-08-17T02:46:22.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:22.920+0000 7f77a4400640 1 -- 172.21.15.16:0/887027163 wait complete. 2024-08-17T02:46:22.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-17T02:46:23.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.086+0000 7ff4ad800640 1 Processor -- start 2024-08-17T02:46:23.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.086+0000 7ff4ad800640 1 -- start start 2024-08-17T02:46:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.087+0000 7ff4ad800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a81401b0 0x7ff4a80caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.087+0000 7ff4ad800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7ff4a80b1440 0x7ff4a80c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.087+0000 7ff4ad800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a80caa80 con 0x7ff4a8140580 2024-08-17T02:46:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.087+0000 7ff4ad800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a80cb450 con 0x7ff4a80b1440 2024-08-17T02:46:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.087+0000 7ff4ad800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80c9e70 con 0x7ff4a81401b0 2024-08-17T02:46:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.088+0000 7ff49fe00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff4a8140580 0x7ff4a80b0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57312/0 (socket says 172.21.15.16:57312) 2024-08-17T02:46:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.088+0000 7ff49fe00640 1 -- 172.21.15.16:0/1113413648 learned_addr learned my addr 172.21.15.16:0/1113413648 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:23.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.088+0000 7ff4a7400640 1 --2- 172.21.15.16:0/1113413648 >> v2:172.21.15.142:3300/0 conn(0x7ff4a80b1440 0x7ff4a80c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:23.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.088+0000 7ff4a6a00640 1 --2- 172.21.15.16:0/1113413648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a81401b0 0x7ff4a80caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.088+0000 7ff4a6000640 1 -- 172.21.15.16:0/1113413648 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994185090 0 0) 0x7ff4a80caa80 con 0x7ff4a8140580 2024-08-17T02:46:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.088+0000 7ff4a6000640 1 -- 172.21.15.16:0/1113413648 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff48c0034e0 con 0x7ff4a8140580 2024-08-17T02:46:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.088+0000 7ff4a7400640 1 -- 172.21.15.16:0/1113413648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a81401b0 msgr2=0x7ff4a80caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.088+0000 7ff4a7400640 1 --2- 172.21.15.16:0/1113413648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a81401b0 0x7ff4a80caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.088+0000 7ff4a7400640 1 -- 172.21.15.16:0/1113413648 >> v1:172.21.15.16:6789/0 conn(0x7ff4a8140580 legacy=0x7ff4a80b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.088+0000 7ff4a7400640 1 -- 172.21.15.16:0/1113413648 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a80ca150 con 0x7ff4a80b1440 2024-08-17T02:46:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.089+0000 7ff4a7400640 1 --2- 172.21.15.16:0/1113413648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a80b1440 0x7ff4a80c9040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009e60 tx=0x7ff4a0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0f4db709315566f server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.090+0000 7ff4a6000640 1 -- 172.21.15.16:0/1113413648 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0007550 con 0x7ff4a80b1440 2024-08-17T02:46:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.090+0000 7ff4a6000640 1 -- 172.21.15.16:0/1113413648 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a002d050 con 0x7ff4a80b1440 2024-08-17T02:46:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.090+0000 7ff4a6000640 1 -- 172.21.15.16:0/1113413648 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0007920 con 0x7ff4a80b1440 2024-08-17T02:46:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.090+0000 7ff4ad800640 1 -- 172.21.15.16:0/1113413648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a80b1440 msgr2=0x7ff4a80c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.090+0000 7ff4ad800640 1 --2- 172.21.15.16:0/1113413648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a80b1440 0x7ff4a80c9040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009e60 tx=0x7ff4a0009780 comp rx=0 tx=0).stop 2024-08-17T02:46:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.090+0000 7ff4ad800640 1 -- 172.21.15.16:0/1113413648 shutdown_connections 2024-08-17T02:46:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.090+0000 7ff4ad800640 1 --2- 172.21.15.16:0/1113413648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a80b1440 0x7ff4a80c9040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.090+0000 7ff4ad800640 1 --2- 172.21.15.16:0/1113413648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a81401b0 0x7ff4a80caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.090+0000 7ff4ad800640 1 -- 172.21.15.16:0/1113413648 >> 172.21.15.16:0/1113413648 conn(0x7ff4a80cb9b0 msgr2=0x7ff4a80cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.090+0000 7ff4ad800640 1 -- 172.21.15.16:0/1113413648 shutdown_connections 2024-08-17T02:46:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.090+0000 7ff4ad800640 1 -- 172.21.15.16:0/1113413648 wait complete. 2024-08-17T02:46:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.092+0000 7ff4ad800640 1 Processor -- start 2024-08-17T02:46:23.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.092+0000 7ff4ad800640 1 -- start start 2024-08-17T02:46:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.093+0000 7ff4ad800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a80b1440 0x7ff4a80c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.093+0000 7ff4ad800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a81401b0 0x7ff4a80a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.093+0000 7ff4ad800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a8140580 0x7ff4a80ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.093+0000 7ff4ad800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80ac480 con 0x7ff4a81401b0 2024-08-17T02:46:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.093+0000 7ff4ad800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a813d0d0 con 0x7ff4a8140580 2024-08-17T02:46:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.093+0000 7ff4ad800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80ab870 con 0x7ff4a80b1440 2024-08-17T02:46:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.093+0000 7ff49fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a81401b0 0x7ff4a80a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.093+0000 7ff49fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a81401b0 0x7ff4a80a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43198/0 (socket says 172.21.15.16:43198) 2024-08-17T02:46:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.093+0000 7ff49fe00640 1 -- 172.21.15.16:0/4044192943 learned_addr learned my addr 172.21.15.16:0/4044192943 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.093+0000 7ff4a7400640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a8140580 0x7ff4a80ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.093+0000 7ff4a6a00640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a80b1440 0x7ff4a80c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:23.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.094+0000 7ff49fe00640 1 -- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a80b1440 msgr2=0x7ff4a80c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.094+0000 7ff49fe00640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a80b1440 0x7ff4a80c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.094+0000 7ff49fe00640 1 -- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a8140580 msgr2=0x7ff4a80ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.094+0000 7ff49fe00640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a8140580 0x7ff4a80ac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.094+0000 7ff49fe00640 1 -- 172.21.15.16:0/4044192943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a0009a20 con 0x7ff4a81401b0 2024-08-17T02:46:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.094+0000 7ff4a6a00640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a80b1440 0x7ff4a80c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:23.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.094+0000 7ff49fe00640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a81401b0 0x7ff4a80a98e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c5b0 tx=0x7ff490007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.094+0000 7ff49e000640 1 -- 172.21.15.16:0/4044192943 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490005aa0 con 0x7ff4a81401b0 2024-08-17T02:46:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.094+0000 7ff49e000640 1 -- 172.21.15.16:0/4044192943 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff490005c00 con 0x7ff4a81401b0 2024-08-17T02:46:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.094+0000 7ff4ad800640 1 -- 172.21.15.16:0/4044192943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a80abb50 con 0x7ff4a81401b0 2024-08-17T02:46:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.094+0000 7ff49e000640 1 -- 172.21.15.16:0/4044192943 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4900070f0 con 0x7ff4a81401b0 2024-08-17T02:46:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.094+0000 7ff4ad800640 1 -- 172.21.15.16:0/4044192943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a80ac110 con 0x7ff4a81401b0 2024-08-17T02:46:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.095+0000 7ff4ad800640 1 -- 172.21.15.16:0/4044192943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a80ac630 con 0x7ff4a81401b0 2024-08-17T02:46:23.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.096+0000 7ff49e000640 1 -- 172.21.15.16:0/4044192943 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7ff490007520 con 0x7ff4a81401b0 2024-08-17T02:46:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.097+0000 7ff49e000640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff484065150 0x7ff484067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.097+0000 7ff49e000640 1 -- 172.21.15.16:0/4044192943 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff490081f90 con 0x7ff4a81401b0 2024-08-17T02:46:23.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.097+0000 7ff4a6a00640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff484065150 0x7ff484067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:23.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.098+0000 7ff4a6a00640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff484065150 0x7ff484067600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff498009cf0 tx=0x7ff498008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:23.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.101+0000 7ff49e000640 1 -- 172.21.15.16:0/4044192943 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff4a80ac630 con 0x7ff4a81401b0 2024-08-17T02:46:23.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.323+0000 7ff4ad800640 1 -- 172.21.15.16:0/4044192943 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff4a8064db0 con 0x7ff4a81401b0 2024-08-17T02:46:23.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.323+0000 7ff49e000640 1 -- 172.21.15.16:0/4044192943 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff490050030 con 0x7ff4a81401b0 2024-08-17T02:46:23.325 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-17T02:46:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.325+0000 7ff4ad800640 1 -- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff484065150 msgr2=0x7ff484067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.325+0000 7ff4ad800640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff484065150 0x7ff484067600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff498009cf0 tx=0x7ff498008040 comp rx=0 tx=0).stop 2024-08-17T02:46:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.325+0000 7ff4ad800640 1 -- 172.21.15.16:0/4044192943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a81401b0 msgr2=0x7ff4a80a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.325+0000 7ff4ad800640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a81401b0 0x7ff4a80a98e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff49000c5b0 tx=0x7ff490007d70 comp rx=0 tx=0).stop 2024-08-17T02:46:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.325+0000 7ff4ad800640 1 -- 172.21.15.16:0/4044192943 shutdown_connections 2024-08-17T02:46:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.325+0000 7ff4ad800640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff484065150 0x7ff484067600 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.325+0000 7ff4ad800640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a8140580 0x7ff4a80ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.325+0000 7ff4ad800640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4a81401b0 0x7ff4a80a98e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.325+0000 7ff4ad800640 1 --2- 172.21.15.16:0/4044192943 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4a80b1440 0x7ff4a80c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.325+0000 7ff4ad800640 1 -- 172.21.15.16:0/4044192943 >> 172.21.15.16:0/4044192943 conn(0x7ff4a80cb9b0 msgr2=0x7ff4a813eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.326+0000 7ff4ad800640 1 -- 172.21.15.16:0/4044192943 shutdown_connections 2024-08-17T02:46:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:23.326+0000 7ff4ad800640 1 -- 172.21.15.16:0/4044192943 wait complete. 2024-08-17T02:46:23.341 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-08-17T02:46:23.341 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-17T02:46:23.341 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-08-17T02:46:23.341 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-17T02:46:23.341 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-17T02:46:23.341 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-08-17T02:46:23.342 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-08-17T02:46:23.342 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-17T02:46:23.342 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-17T02:46:23.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:23.342 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-08-17T02:46:23.378 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:46:23.378 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:46:23.378 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:46:23.378 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:23.378 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-08-17T02:46:23.378 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:23.378 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:46:23.378 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42595sec preferred_lft 42595sec 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:46:23.379 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 4e:62:eb:b3:e9:23 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::a4bb:7ff:fefc:5693/64 scope link 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether fa:8a:ae:13:08:49 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::f88a:aeff:fe13:849/64 scope link 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 4e:62:eb:b3:e9:23 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::4c62:ebff:feb3:e923/64 scope link 2024-08-17T02:46:23.380 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:23.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:23.381 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-17T02:46:23.471 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-08-17T02:46:23.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:23.475 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-08-17T02:46:23.511 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-08-17T02:46:23.511 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-17T02:46:23.511 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-08-17T02:46:23.512 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-08-17T02:46:23.512 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-17T02:46:23.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:23.512 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2024-08-17T02:46:23.512 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-08-17T02:46:23.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-08-17T02:46:23.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-08-17T02:46:23.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-08-17T02:46:23.512 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-08-17T02:46:23.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:23.658 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-08-17T02:46:23.693 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-17T02:46:23.693 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-08-17T02:46:23.693 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-17T02:46:23.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:23.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-08-17T02:46:23.791 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:46:23.791 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-17T02:46:23.827 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-17T02:46:23.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:46:23.827 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-17T02:46:23.902 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-08-17T02:46:23.902 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-17T02:46:23.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:46:24.024 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:46:24.024 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-08-17T02:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-08-17T02:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-08-17T02:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778.cephfs=/ 2024-08-17T02:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.16:3300/172.21.15.142:3300/172.21.15.142:3301" will pass to kernel 2024-08-17T02:46:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-08-17T02:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-08-17T02:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-17T02:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-08-17T02:46:24.053 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-17T02:46:24.056 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-17T02:46:24.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:24.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-17T02:46:24.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-08-17T02:46:24.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.265+0000 7f1ece800640 1 Processor -- start 2024-08-17T02:46:24.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.265+0000 7f1ece800640 1 -- start start 2024-08-17T02:46:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.266+0000 7f1ece800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ec8159ca0 0x7f1ec80ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.266+0000 7f1ece800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1ec80ccca0 0x7f1ec8162860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.266+0000 7f1ece800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec80cadc0 con 0x7f1ec80ca8a0 2024-08-17T02:46:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.266+0000 7f1ece800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec80cd6e0 con 0x7f1ec8159ca0 2024-08-17T02:46:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.266+0000 7f1ece800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec80cb060 con 0x7f1ec80ccca0 2024-08-17T02:46:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.266+0000 7f1ecd400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ec8159ca0 0x7f1ec80ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:24.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.266+0000 7f1ecd400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ec8159ca0 0x7f1ec80ca360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34302/0 (socket says 172.21.15.16:34302) 2024-08-17T02:46:24.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.266+0000 7f1ecd400640 1 -- 172.21.15.16:0/703456376 learned_addr learned my addr 172.21.15.16:0/703456376 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:24.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.266+0000 7f1ecde00640 1 --2- 172.21.15.16:0/703456376 >> v2:172.21.15.142:3300/0 conn(0x7f1ec80ccca0 0x7f1ec8162860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:24.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.267+0000 7f1ecd400640 1 -- 172.21.15.16:0/703456376 >> v2:172.21.15.142:3300/0 conn(0x7f1ec80ccca0 msgr2=0x7f1ec8162860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.267+0000 7f1ecd400640 1 --2- 172.21.15.16:0/703456376 >> v2:172.21.15.142:3300/0 conn(0x7f1ec80ccca0 0x7f1ec8162860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.267+0000 7f1ecd400640 1 -- 172.21.15.16:0/703456376 >> v1:172.21.15.142:6789/0 conn(0x7f1ec80ca8a0 legacy=0x7f1ec80cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.267+0000 7f1ecd400640 1 -- 172.21.15.16:0/703456376 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec80cb300 con 0x7f1ec8159ca0 2024-08-17T02:46:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.267+0000 7f1ecde00640 1 --2- 172.21.15.16:0/703456376 >> v2:172.21.15.142:3300/0 conn(0x7f1ec80ccca0 0x7f1ec8162860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:24.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.268+0000 7f1ecd400640 1 --2- 172.21.15.16:0/703456376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8159ca0 0x7f1ec80ca360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80b26e0 tx=0x7f1ebc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0cb7351d6efcb server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.268+0000 7f1ec7e00640 1 -- 172.21.15.16:0/703456376 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc02ae80 con 0x7f1ec8159ca0 2024-08-17T02:46:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.268+0000 7f1ec7e00640 1 -- 172.21.15.16:0/703456376 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc032440 con 0x7f1ec8159ca0 2024-08-17T02:46:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.268+0000 7f1ec7e00640 1 -- 172.21.15.16:0/703456376 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc032800 con 0x7f1ec8159ca0 2024-08-17T02:46:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.268+0000 7f1ece800640 1 -- 172.21.15.16:0/703456376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8159ca0 msgr2=0x7f1ec80ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.268+0000 7f1ece800640 1 --2- 172.21.15.16:0/703456376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8159ca0 0x7f1ec80ca360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80b26e0 tx=0x7f1ebc0091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:24.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.269+0000 7f1ece800640 1 -- 172.21.15.16:0/703456376 shutdown_connections 2024-08-17T02:46:24.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.269+0000 7f1ece800640 1 --2- 172.21.15.16:0/703456376 >> v2:172.21.15.142:3300/0 conn(0x7f1ec80ccca0 0x7f1ec8162860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:24.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.269+0000 7f1ece800640 1 --2- 172.21.15.16:0/703456376 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8159ca0 0x7f1ec80ca360 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:24.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.269+0000 7f1ece800640 1 -- 172.21.15.16:0/703456376 >> 172.21.15.16:0/703456376 conn(0x7f1ec8098e00 msgr2=0x7f1ec806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:24.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.269+0000 7f1ece800640 1 -- 172.21.15.16:0/703456376 shutdown_connections 2024-08-17T02:46:24.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.269+0000 7f1ece800640 1 -- 172.21.15.16:0/703456376 wait complete. 2024-08-17T02:46:24.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.270+0000 7f1ece800640 1 Processor -- start 2024-08-17T02:46:24.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.270+0000 7f1ece800640 1 -- start start 2024-08-17T02:46:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.271+0000 7f1ece800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ec80ca8a0 0x7f1ec80c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:24.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.271+0000 7f1ece800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ec80ccca0 0x7f1ec80c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.271+0000 7f1ece800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8159ca0 0x7f1ec8168640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.271+0000 7f1ece800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c7f20 con 0x7f1ec8159ca0 2024-08-17T02:46:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.271+0000 7f1ece800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec809a4d0 con 0x7f1ec80ca8a0 2024-08-17T02:46:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.271+0000 7f1ece800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c7100 con 0x7f1ec80ccca0 2024-08-17T02:46:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.271+0000 7f1ecde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8159ca0 0x7f1ec8168640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.271+0000 7f1ecde00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8159ca0 0x7f1ec8168640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34318/0 (socket says 172.21.15.16:34318) 2024-08-17T02:46:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.271+0000 7f1ecde00640 1 -- 172.21.15.16:0/2418968095 learned_addr learned my addr 172.21.15.16:0/2418968095 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.272+0000 7f1ecd400640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ec80ca8a0 0x7f1ec80c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:24.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.272+0000 7f1ecca00640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ec80ccca0 0x7f1ec80c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.272+0000 7f1ecde00640 1 -- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ec80ccca0 msgr2=0x7f1ec80c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.272+0000 7f1ecde00640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ec80ccca0 0x7f1ec80c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:24.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.272+0000 7f1ecde00640 1 -- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ec80ca8a0 msgr2=0x7f1ec80c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.272+0000 7f1ecde00640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ec80ca8a0 0x7f1ec80c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.272+0000 7f1ecde00640 1 -- 172.21.15.16:0/2418968095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc009450 con 0x7f1ec8159ca0 2024-08-17T02:46:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.272+0000 7f1ecca00640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ec80ccca0 0x7f1ec80c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.272+0000 7f1ecde00640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8159ca0 0x7f1ec8168640 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000d150 tx=0x7f1ec0009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.272+0000 7f1ec4c00640 1 -- 172.21.15.16:0/2418968095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0015400 con 0x7f1ec8159ca0 2024-08-17T02:46:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.273+0000 7f1ec4c00640 1 -- 172.21.15.16:0/2418968095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec000f040 con 0x7f1ec8159ca0 2024-08-17T02:46:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.273+0000 7f1ec4c00640 1 -- 172.21.15.16:0/2418968095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec00157b0 con 0x7f1ec8159ca0 2024-08-17T02:46:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.273+0000 7f1ece800640 1 -- 172.21.15.16:0/2418968095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec8168b80 con 0x7f1ec8159ca0 2024-08-17T02:46:24.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.273+0000 7f1ece800640 1 -- 172.21.15.16:0/2418968095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec80c7680 con 0x7f1ec8159ca0 2024-08-17T02:46:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.274+0000 7f1ece800640 1 -- 172.21.15.16:0/2418968095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec809a660 con 0x7f1ec8159ca0 2024-08-17T02:46:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.274+0000 7f1ec4c00640 1 -- 172.21.15.16:0/2418968095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1ec0015ba0 con 0x7f1ec8159ca0 2024-08-17T02:46:24.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.275+0000 7f1ec4c00640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1ea8065150 0x7f1ea8067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:24.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.275+0000 7f1ec4c00640 1 -- 172.21.15.16:0/2418968095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1ec0082d40 con 0x7f1ec8159ca0 2024-08-17T02:46:24.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.276+0000 7f1ecd400640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1ea8065150 0x7f1ea8067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:24.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.276+0000 7f1ecd400640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1ea8065150 0x7f1ea8067600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc034000 tx=0x7f1ebc032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:24.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.279+0000 7f1ec4c00640 1 -- 172.21.15.16:0/2418968095 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1ec809a660 con 0x7f1ec8159ca0 2024-08-17T02:46:24.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.502+0000 7f1ece800640 1 -- 172.21.15.16:0/2418968095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ec809a610 con 0x7f1ec8159ca0 2024-08-17T02:46:24.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.503+0000 7f1ec4c00640 1 -- 172.21.15.16:0/2418968095 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1ec0050d60 con 0x7f1ec8159ca0 2024-08-17T02:46:24.504 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-17T02:46:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.505+0000 7f1ece800640 1 -- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1ea8065150 msgr2=0x7f1ea8067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:24.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.505+0000 7f1ece800640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1ea8065150 0x7f1ea8067600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc034000 tx=0x7f1ebc032000 comp rx=0 tx=0).stop 2024-08-17T02:46:24.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.505+0000 7f1ece800640 1 -- 172.21.15.16:0/2418968095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8159ca0 msgr2=0x7f1ec8168640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:24.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.505+0000 7f1ece800640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8159ca0 0x7f1ec8168640 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000d150 tx=0x7f1ec0009a50 comp rx=0 tx=0).stop 2024-08-17T02:46:24.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.505+0000 7f1ece800640 1 -- 172.21.15.16:0/2418968095 shutdown_connections 2024-08-17T02:46:24.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.505+0000 7f1ece800640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1ea8065150 0x7f1ea8067600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:24.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.505+0000 7f1ece800640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ec8159ca0 0x7f1ec8168640 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:24.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.505+0000 7f1ece800640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1ec80ccca0 0x7f1ec80c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:24.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.505+0000 7f1ece800640 1 --2- 172.21.15.16:0/2418968095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ec80ca8a0 0x7f1ec80c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:24.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.505+0000 7f1ece800640 1 -- 172.21.15.16:0/2418968095 >> 172.21.15.16:0/2418968095 conn(0x7f1ec8098e00 msgr2=0x7f1ec8158440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:24.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.505+0000 7f1ece800640 1 -- 172.21.15.16:0/2418968095 shutdown_connections 2024-08-17T02:46:24.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:24.505+0000 7f1ece800640 1 -- 172.21.15.16:0/2418968095 wait complete. 2024-08-17T02:46:24.521 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-08-17T02:46:24.521 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-17T02:46:24.522 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi142.front.sepia.ceph.com 2024-08-17T02:46:24.522 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-08-17T02:46:24.522 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-08-17T02:46:24.522 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-08-17T02:46:24.522 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-08-17T02:46:24.522 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-17T02:46:24.522 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-17T02:46:24.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:24.522 DEBUG:teuthology.orchestra.run.smithi142:> ip addr 2024-08-17T02:46:24.562 INFO:teuthology.orchestra.run.smithi142.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-17T02:46:24.562 INFO:teuthology.orchestra.run.smithi142.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-17T02:46:24.562 INFO:teuthology.orchestra.run.smithi142.stdout: inet 127.0.0.1/8 scope host lo 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 ::1/128 scope host 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3c brd ff:ff:ff:ff:ff:ff 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f0 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:d9:36:3d brd ff:ff:ff:ff:ff:ff 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout: altname enp7s0f1 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0c brd ff:ff:ff:ff:ff:ff 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-17T02:46:24.563 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 0c:c4:7a:88:6e:0d brd ff:ff:ff:ff:ff:ff 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout: inet 172.21.15.142/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft 42593sec preferred_lft 42593sec 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::ec4:7aff:fe88:6e0d/64 scope link 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 42:93:50:74:3a:42 brd ff:ff:ff:ff:ff:ff 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::7054:9fff:feff:9dd/64 scope link 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 42:93:50:74:3a:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-17T02:46:24.564 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::4093:50ff:fe74:3a42/64 scope link 2024-08-17T02:46:24.565 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:24.565 INFO:teuthology.orchestra.run.smithi142.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-08-17T02:46:24.565 INFO:teuthology.orchestra.run.smithi142.stdout: link/ether 6a:37:c9:4a:bc:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-17T02:46:24.565 INFO:teuthology.orchestra.run.smithi142.stdout: inet6 fe80::6837:c9ff:fe4a:bcfa/64 scope link 2024-08-17T02:46:24.565 INFO:teuthology.orchestra.run.smithi142.stdout: valid_lft forever preferred_lft forever 2024-08-17T02:46:24.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:24.565 DEBUG:teuthology.orchestra.run.smithi142:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-17T02:46:24.659 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-08-17T02:46:24.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:24.663 DEBUG:teuthology.orchestra.run.smithi142:> route 2024-08-17T02:46:24.699 INFO:teuthology.orchestra.run.smithi142.stdout:Kernel IP routing table 2024-08-17T02:46:24.700 INFO:teuthology.orchestra.run.smithi142.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-17T02:46:24.700 INFO:teuthology.orchestra.run.smithi142.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-17T02:46:24.700 INFO:teuthology.orchestra.run.smithi142.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-17T02:46:24.700 INFO:teuthology.orchestra.run.smithi142.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-17T02:46:24.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:24.700 DEBUG:teuthology.orchestra.run.smithi142:> (cd / && exec bash -c ' 2024-08-17T02:46:24.700 DEBUG:teuthology.orchestra.run.smithi142:> set -e 2024-08-17T02:46:24.700 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-17T02:46:24.701 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-17T02:46:24.701 DEBUG:teuthology.orchestra.run.smithi142:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-17T02:46:24.701 DEBUG:teuthology.orchestra.run.smithi142:> ') 2024-08-17T02:46:24.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:24.849 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-08-17T02:46:24.884 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-17T02:46:24.884 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-08-17T02:46:24.884 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-08-17T02:46:24.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:24.885 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link set brx.0 up 2024-08-17T02:46:24.981 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:46:24.982 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-08-17T02:46:25.017 INFO:teuthology.orchestra.run.smithi142.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-08-17T02:46:25.017 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-17T02:46:25.017 DEBUG:teuthology.orchestra.run.smithi142:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-08-17T02:46:25.091 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-08-17T02:46:25.092 INFO:teuthology.orchestra.run:Running command with timeout 1800 2024-08-17T02:46:25.092 DEBUG:teuthology.orchestra.run.smithi142:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:46:25.212 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:46:25.212 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-08-17T02:46:25.239 INFO:teuthology.orchestra.run.smithi142.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-08-17T02:46:25.239 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-08-17T02:46:25.239 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.142:3300,172.21.15.142:3301" 2024-08-17T02:46:25.239 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: trying mount with new device syntax: 1@4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778.cephfs=/ 2024-08-17T02:46:25.240 INFO:teuthology.orchestra.run.smithi142.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.16:3300/172.21.15.142:3300/172.21.15.142:3301" will pass to kernel 2024-08-17T02:46:25.240 INFO:teuthology.orchestra.run.smithi142.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-08-17T02:46:25.240 INFO:teuthology.orchestra.run.smithi142.stdout: You just mounted a file system that supports labels which does not 2024-08-17T02:46:25.240 INFO:teuthology.orchestra.run.smithi142.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-17T02:46:25.240 INFO:teuthology.orchestra.run.smithi142.stdout: applications will generate AVC messages and not be allowed access to 2024-08-17T02:46:25.240 INFO:teuthology.orchestra.run.smithi142.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-17T02:46:25.243 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-17T02:46:25.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:46:25.243 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-08-17T02:46:25.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-08-17T02:46:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.461+0000 7f133ee00640 1 Processor -- start 2024-08-17T02:46:25.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.461+0000 7f133ee00640 1 -- start start 2024-08-17T02:46:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.463+0000 7f133ee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380aaed0 0x7f13380b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.463+0000 7f133ee00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13380b0830 con 0x7f133815a010 2024-08-17T02:46:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.463+0000 7f133ee00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13380b1690 con 0x7f13380b3430 2024-08-17T02:46:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.463+0000 7f133ee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13380ad5b0 con 0x7f13380aaed0 2024-08-17T02:46:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.463+0000 7f133da00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f133815a010 0x7f13380aa7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54190/0 (socket says 172.21.15.16:54190) 2024-08-17T02:46:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.463+0000 7f133da00640 1 -- 172.21.15.16:0/3070067465 learned_addr learned my addr 172.21.15.16:0/3070067465 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.463+0000 7f133d000640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380aaed0 0x7f13380b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.463+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207101365 0 0) 0x7f13380b0830 con 0x7f133815a010 2024-08-17T02:46:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.463+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13200034e0 con 0x7f133815a010 2024-08-17T02:46:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1278589673 0 0) 0x7f13200034e0 con 0x7f133815a010 2024-08-17T02:46:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380aaed0 msgr2=0x7f13380b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380aaed0 0x7f13380b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 >> v1:172.21.15.142:6789/0 conn(0x7f13380b3430 legacy=0x7f1338160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13380b3870 con 0x7f133815a010 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7f132c001470 con 0x7f133815a010 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 >> v1:172.21.15.16:6789/0 conn(0x7f133815a010 legacy=0x7f13380aa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1320003a00 0x7f1320003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380aaed0 0x7f1320004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1320004cc0 0x7f1320005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13380ad5b0 con 0x7f1320003a00 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132000ba80 con 0x7f1320004cc0 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132000bc40 con 0x7f13380aaed0 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f133da00640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1320003a00 0x7f1320003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f133e400640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1320004cc0 0x7f1320005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.464+0000 7f133d000640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380aaed0 0x7f1320004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.465+0000 7f133da00640 1 -- 172.21.15.16:0/3070067465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380aaed0 msgr2=0x7f1320004780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.465+0000 7f133da00640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380aaed0 0x7f1320004780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.465+0000 7f133da00640 1 -- 172.21.15.16:0/3070067465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1320004cc0 msgr2=0x7f1320005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.465+0000 7f133da00640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1320004cc0 0x7f1320005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.465+0000 7f133da00640 1 -- 172.21.15.16:0/3070067465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13200034e0 con 0x7f1320003a00 2024-08-17T02:46:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.465+0000 7f133e400640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1320004cc0 0x7f1320005130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.465+0000 7f133da00640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1320003a00 0x7f1320003dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f132c007fd0 tx=0x7f13380ad490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ceca2ddf3cb5ca server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.466+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c005890 con 0x7f1320003a00 2024-08-17T02:46:25.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.466+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c02e070 con 0x7f1320003a00 2024-08-17T02:46:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.466+0000 7f1337e00640 1 -- 172.21.15.16:0/3070067465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c005a50 con 0x7f1320003a00 2024-08-17T02:46:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.466+0000 7f133ee00640 1 -- 172.21.15.16:0/3070067465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1320003a00 msgr2=0x7f1320003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.466+0000 7f133ee00640 1 --2- 172.21.15.16:0/3070067465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1320003a00 0x7f1320003dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f132c007fd0 tx=0x7f13380ad490 comp rx=0 tx=0).stop 2024-08-17T02:46:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.466+0000 7f133e400640 1 -- 172.21.15.16:0/3070067465 reap_dead start 2024-08-17T02:46:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.466+0000 7f133ee00640 1 -- 172.21.15.16:0/3070067465 shutdown_connections 2024-08-17T02:46:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.466+0000 7f133ee00640 1 -- 172.21.15.16:0/3070067465 >> 172.21.15.16:0/3070067465 conn(0x7f1338096a70 msgr2=0x7f1338096e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.467+0000 7f133ee00640 1 -- 172.21.15.16:0/3070067465 shutdown_connections 2024-08-17T02:46:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.467+0000 7f133ee00640 1 -- 172.21.15.16:0/3070067465 wait complete. 2024-08-17T02:46:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.468+0000 7f133ee00640 1 Processor -- start 2024-08-17T02:46:25.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.468+0000 7f133ee00640 1 -- start start 2024-08-17T02:46:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.469+0000 7f133ee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1320003a00 0x7f133809b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.469+0000 7f133ee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13380ccb50 0x7f13380ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.469+0000 7f133ee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380ccfa0 0x7f13380aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.469+0000 7f133ee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133809a1e0 con 0x7f13380ccb50 2024-08-17T02:46:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.469+0000 7f133ee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133809a4c0 con 0x7f1320003a00 2024-08-17T02:46:25.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.469+0000 7f133ee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133809a7a0 con 0x7f13380ccfa0 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.469+0000 7f133d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13380ccb50 0x7f13380ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.469+0000 7f133d000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13380ccb50 0x7f13380ae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34332/0 (socket says 172.21.15.16:34332) 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.469+0000 7f133d000640 1 -- 172.21.15.16:0/1589471528 learned_addr learned my addr 172.21.15.16:0/1589471528 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.469+0000 7f133da00640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1320003a00 0x7f133809b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.469+0000 7f133e400640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380ccfa0 0x7f13380aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.470+0000 7f133d000640 1 -- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380ccfa0 msgr2=0x7f13380aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.470+0000 7f133d000640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380ccfa0 0x7f13380aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.470+0000 7f133d000640 1 -- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1320003a00 msgr2=0x7f133809b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.470+0000 7f133d000640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1320003a00 0x7f133809b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.470+0000 7f133d000640 1 -- 172.21.15.16:0/1589471528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133809aa80 con 0x7f13380ccb50 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.470+0000 7f133d000640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13380ccb50 0x7f13380ae6f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1324011b40 tx=0x7f13240090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.470+0000 7f1334c00640 1 -- 172.21.15.16:0/1589471528 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132400faa0 con 0x7f13380ccb50 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.470+0000 7f1334c00640 1 -- 172.21.15.16:0/1589471528 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132400fc00 con 0x7f13380ccb50 2024-08-17T02:46:25.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.470+0000 7f1334c00640 1 -- 172.21.15.16:0/1589471528 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324014440 con 0x7f13380ccb50 2024-08-17T02:46:25.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.470+0000 7f133ee00640 1 -- 172.21.15.16:0/1589471528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133809ad60 con 0x7f13380ccb50 2024-08-17T02:46:25.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.471+0000 7f133ee00640 1 -- 172.21.15.16:0/1589471528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1338099460 con 0x7f13380ccb50 2024-08-17T02:46:25.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.472+0000 7f133ee00640 1 -- 172.21.15.16:0/1589471528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1338063d20 con 0x7f13380ccb50 2024-08-17T02:46:25.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.472+0000 7f1334c00640 1 -- 172.21.15.16:0/1589471528 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1324014830 con 0x7f13380ccb50 2024-08-17T02:46:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.473+0000 7f1334c00640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1318065150 0x7f1318067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.473+0000 7f1334c00640 1 -- 172.21.15.16:0/1589471528 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1324018030 con 0x7f13380ccb50 2024-08-17T02:46:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.474+0000 7f133da00640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1318065150 0x7f1318067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:25.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.474+0000 7f133da00640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1318065150 0x7f1318067600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f132c033000 tx=0x7f132c034000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:25.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.477+0000 7f1334c00640 1 -- 172.21.15.16:0/1589471528 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1324050e60 con 0x7f13380ccb50 2024-08-17T02:46:25.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.696+0000 7f133ee00640 1 -- 172.21.15.16:0/1589471528 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1338071050 con 0x7f13380ccb50 2024-08-17T02:46:25.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.908+0000 7f1334c00640 1 -- 172.21.15.16:0/1589471528 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 136+0+0 (secure 0 0 0) 0x7f13240220c0 con 0x7f13380ccb50 2024-08-17T02:46:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.910+0000 7f133ee00640 1 -- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1318065150 msgr2=0x7f1318067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.910+0000 7f133ee00640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1318065150 0x7f1318067600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f132c033000 tx=0x7f132c034000 comp rx=0 tx=0).stop 2024-08-17T02:46:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.910+0000 7f133ee00640 1 -- 172.21.15.16:0/1589471528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13380ccb50 msgr2=0x7f13380ae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.910+0000 7f133ee00640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13380ccb50 0x7f13380ae6f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1324011b40 tx=0x7f13240090f0 comp rx=0 tx=0).stop 2024-08-17T02:46:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.910+0000 7f133ee00640 1 -- 172.21.15.16:0/1589471528 shutdown_connections 2024-08-17T02:46:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.910+0000 7f133ee00640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1318065150 0x7f1318067600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.910+0000 7f133ee00640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f13380ccfa0 0x7f13380aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:25.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.910+0000 7f133ee00640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13380ccb50 0x7f13380ae6f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:25.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.910+0000 7f133ee00640 1 --2- 172.21.15.16:0/1589471528 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1320003a00 0x7f133809b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:25.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.910+0000 7f133ee00640 1 -- 172.21.15.16:0/1589471528 >> 172.21.15.16:0/1589471528 conn(0x7f1338096a70 msgr2=0x7f133806b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:25.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.911+0000 7f133ee00640 1 -- 172.21.15.16:0/1589471528 shutdown_connections 2024-08-17T02:46:25.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:25.911+0000 7f133ee00640 1 -- 172.21.15.16:0/1589471528 wait complete. 2024-08-17T02:46:25.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-08-17T02:46:26.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.076+0000 7fde99800640 1 Processor -- start 2024-08-17T02:46:26.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.076+0000 7fde99800640 1 -- start start 2024-08-17T02:46:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.077+0000 7fde99800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde941401b0 0x7fde940caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.077+0000 7fde99800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde940caa80 con 0x7fde94140580 2024-08-17T02:46:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.077+0000 7fde99800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde940c9e70 con 0x7fde940b1440 2024-08-17T02:46:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.077+0000 7fde99800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde940cb450 con 0x7fde941401b0 2024-08-17T02:46:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.078+0000 7fde92a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde941401b0 0x7fde940caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.078+0000 7fde92a00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde941401b0 0x7fde940caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34348/0 (socket says 172.21.15.16:34348) 2024-08-17T02:46:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.078+0000 7fde92000640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fde94140580 0x7fde940b0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54196/0 (socket says 172.21.15.16:54196) 2024-08-17T02:46:26.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.078+0000 7fde92a00640 1 -- 172.21.15.16:0/21886973 learned_addr learned my addr 172.21.15.16:0/21886973 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.078+0000 7fde92a00640 1 -- 172.21.15.16:0/21886973 >> v1:172.21.15.142:6789/0 conn(0x7fde940b1440 legacy=0x7fde940c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.078+0000 7fde92a00640 1 -- 172.21.15.16:0/21886973 >> v1:172.21.15.16:6789/0 conn(0x7fde94140580 legacy=0x7fde940b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.078+0000 7fde92a00640 1 -- 172.21.15.16:0/21886973 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde940ca150 con 0x7fde941401b0 2024-08-17T02:46:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.078+0000 7fde91600640 1 -- 172.21.15.16:0/21886973 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549898729 0 0) 0x7fde940caa80 con 0x7fde94140580 2024-08-17T02:46:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.079+0000 7fde92a00640 1 --2- 172.21.15.16:0/21886973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde941401b0 0x7fde940caf10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fde84009910 tx=0x7fde840091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3159530879500fe0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.079+0000 7fde91600640 1 -- 172.21.15.16:0/21886973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8402ae60 con 0x7fde941401b0 2024-08-17T02:46:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.080+0000 7fde91600640 1 -- 172.21.15.16:0/21886973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde84032440 con 0x7fde941401b0 2024-08-17T02:46:26.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.080+0000 7fde91600640 1 -- 172.21.15.16:0/21886973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde840327e0 con 0x7fde941401b0 2024-08-17T02:46:26.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.080+0000 7fde99800640 1 -- 172.21.15.16:0/21886973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde941401b0 msgr2=0x7fde940caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.080+0000 7fde99800640 1 --2- 172.21.15.16:0/21886973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde941401b0 0x7fde940caf10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fde84009910 tx=0x7fde840091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.080+0000 7fde99800640 1 -- 172.21.15.16:0/21886973 shutdown_connections 2024-08-17T02:46:26.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.080+0000 7fde99800640 1 --2- 172.21.15.16:0/21886973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde941401b0 0x7fde940caf10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.080+0000 7fde99800640 1 -- 172.21.15.16:0/21886973 >> 172.21.15.16:0/21886973 conn(0x7fde940cb9b0 msgr2=0x7fde940cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:26.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.080+0000 7fde99800640 1 -- 172.21.15.16:0/21886973 shutdown_connections 2024-08-17T02:46:26.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.080+0000 7fde99800640 1 -- 172.21.15.16:0/21886973 wait complete. 2024-08-17T02:46:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.082+0000 7fde99800640 1 Processor -- start 2024-08-17T02:46:26.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.082+0000 7fde99800640 1 -- start start 2024-08-17T02:46:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde99800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde940b1440 0x7fde940a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde99800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde941401b0 0x7fde940a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde99800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde94140580 0x7fde940a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde99800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9413ecb0 con 0x7fde941401b0 2024-08-17T02:46:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde99800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde940a7be0 con 0x7fde94140580 2024-08-17T02:46:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde99800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde94159e00 con 0x7fde940b1440 2024-08-17T02:46:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde92000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde941401b0 0x7fde940a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde92000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde941401b0 0x7fde940a65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34358/0 (socket says 172.21.15.16:34358) 2024-08-17T02:46:26.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde92000640 1 -- 172.21.15.16:0/1138023905 learned_addr learned my addr 172.21.15.16:0/1138023905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde92a00640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde940b1440 0x7fde940a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde93400640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde94140580 0x7fde940a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde92000640 1 -- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde940b1440 msgr2=0x7fde940a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde92000640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde940b1440 0x7fde940a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde92000640 1 -- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde94140580 msgr2=0x7fde940a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde92000640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde94140580 0x7fde940a6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.083+0000 7fde92000640 1 -- 172.21.15.16:0/1138023905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde84009450 con 0x7fde941401b0 2024-08-17T02:46:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.084+0000 7fde92000640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde941401b0 0x7fde940a65c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fde8000c5e0 tx=0x7fde80007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.084+0000 7fde8a000640 1 -- 172.21.15.16:0/1138023905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80007300 con 0x7fde941401b0 2024-08-17T02:46:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.084+0000 7fde8a000640 1 -- 172.21.15.16:0/1138023905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde80007480 con 0x7fde941401b0 2024-08-17T02:46:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.084+0000 7fde8a000640 1 -- 172.21.15.16:0/1138023905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800049f0 con 0x7fde941401b0 2024-08-17T02:46:26.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.084+0000 7fde99800640 1 -- 172.21.15.16:0/1138023905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde940a6170 con 0x7fde941401b0 2024-08-17T02:46:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.084+0000 7fde99800640 1 -- 172.21.15.16:0/1138023905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9415a380 con 0x7fde941401b0 2024-08-17T02:46:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.085+0000 7fde99800640 1 -- 172.21.15.16:0/1138023905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde94064db0 con 0x7fde941401b0 2024-08-17T02:46:26.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.086+0000 7fde8a000640 1 -- 172.21.15.16:0/1138023905 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fde800113a0 con 0x7fde941401b0 2024-08-17T02:46:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.087+0000 7fde8a000640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fde74065180 0x7fde74067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.087+0000 7fde8a000640 1 -- 172.21.15.16:0/1138023905 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fde80081420 con 0x7fde941401b0 2024-08-17T02:46:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.088+0000 7fde92a00640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fde74065180 0x7fde74067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.088+0000 7fde92a00640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fde74065180 0x7fde74067630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fde84034000 tx=0x7fde84032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.091+0000 7fde8a000640 1 -- 172.21.15.16:0/1138023905 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fde8004f570 con 0x7fde941401b0 2024-08-17T02:46:26.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.295+0000 7fde99800640 1 -- 172.21.15.16:0/1138023905 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fde940a7d20 con 0x7fde74065180 2024-08-17T02:46:26.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.297+0000 7fde8a000640 1 -- 172.21.15.16:0/1138023905 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fde940a7d20 con 0x7fde74065180 2024-08-17T02:46:26.299 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-08-17T02:46:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephfs" 2024-08-17T02:46:26.299 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:26.299 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-08-17T02:46:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.299+0000 7fde99800640 1 -- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fde74065180 msgr2=0x7fde74067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.299+0000 7fde99800640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fde74065180 0x7fde74067630 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fde84034000 tx=0x7fde84032000 comp rx=0 tx=0).stop 2024-08-17T02:46:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.299+0000 7fde99800640 1 -- 172.21.15.16:0/1138023905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde941401b0 msgr2=0x7fde940a65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.299+0000 7fde99800640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde941401b0 0x7fde940a65c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fde8000c5e0 tx=0x7fde80007d00 comp rx=0 tx=0).stop 2024-08-17T02:46:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.299+0000 7fde99800640 1 -- 172.21.15.16:0/1138023905 shutdown_connections 2024-08-17T02:46:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.299+0000 7fde99800640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fde74065180 0x7fde74067630 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.299+0000 7fde99800640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde94140580 0x7fde940a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.299+0000 7fde99800640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde941401b0 0x7fde940a65c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.299+0000 7fde99800640 1 --2- 172.21.15.16:0/1138023905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde940b1440 0x7fde940a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.299+0000 7fde99800640 1 -- 172.21.15.16:0/1138023905 >> 172.21.15.16:0/1138023905 conn(0x7fde940cb9b0 msgr2=0x7fde9413ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:26.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.300+0000 7fde99800640 1 -- 172.21.15.16:0/1138023905 shutdown_connections 2024-08-17T02:46:26.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.300+0000 7fde99800640 1 -- 172.21.15.16:0/1138023905 wait complete. 2024-08-17T02:46:26.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-08-17T02:46:26.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.469+0000 7fb7a5c00640 1 Processor -- start 2024-08-17T02:46:26.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.469+0000 7fb7a5c00640 1 -- start start 2024-08-17T02:46:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.470+0000 7fb7a5c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fb7a0158e20 0x7fb7a00ca310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.470+0000 7fb7a5c00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7a00c3390 0x7fb7a00ca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.470+0000 7fb7a5c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7a0088f20 0x7fb7a00cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.470+0000 7fb7a5c00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a00cb540 con 0x7fb7a00c3390 2024-08-17T02:46:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.470+0000 7fb7a5c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a00cf7f0 con 0x7fb7a0158e20 2024-08-17T02:46:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.470+0000 7fb7a5c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a00caef0 con 0x7fb7a0088f20 2024-08-17T02:46:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.470+0000 7fb79e000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7a00c3390 0x7fb7a00ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.470+0000 7fb79e000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7a00c3390 0x7fb7a00ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34372/0 (socket says 172.21.15.16:34372) 2024-08-17T02:46:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.470+0000 7fb79e000640 1 -- 172.21.15.16:0/2779295643 learned_addr learned my addr 172.21.15.16:0/2779295643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.471+0000 7fb79ea00640 1 --2- 172.21.15.16:0/2779295643 >> v2:172.21.15.142:3300/0 conn(0x7fb7a0158e20 0x7fb7a00ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.471+0000 7fb79f400640 1 --2- 172.21.15.16:0/2779295643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7a0088f20 0x7fb7a00cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.471+0000 7fb79e000640 1 -- 172.21.15.16:0/2779295643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7a0088f20 msgr2=0x7fb7a00cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.471+0000 7fb79e000640 1 --2- 172.21.15.16:0/2779295643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7a0088f20 0x7fb7a00cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.471+0000 7fb79e000640 1 -- 172.21.15.16:0/2779295643 >> v2:172.21.15.142:3300/0 conn(0x7fb7a0158e20 msgr2=0x7fb7a00ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.471+0000 7fb79e000640 1 --2- 172.21.15.16:0/2779295643 >> v2:172.21.15.142:3300/0 conn(0x7fb7a0158e20 0x7fb7a00ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.471+0000 7fb79e000640 1 -- 172.21.15.16:0/2779295643 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a00cb1d0 con 0x7fb7a00c3390 2024-08-17T02:46:26.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.472+0000 7fb79e000640 1 --2- 172.21.15.16:0/2779295643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a00c3390 0x7fb7a00ca850 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00644c0 tx=0x7fb78c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfca38f12bc33fe server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.472+0000 7fb79d600640 1 -- 172.21.15.16:0/2779295643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c02add0 con 0x7fb7a00c3390 2024-08-17T02:46:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.472+0000 7fb79d600640 1 -- 172.21.15.16:0/2779295643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c032440 con 0x7fb7a00c3390 2024-08-17T02:46:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.472+0000 7fb79d600640 1 -- 172.21.15.16:0/2779295643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c032820 con 0x7fb7a00c3390 2024-08-17T02:46:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.473+0000 7fb7a5c00640 1 -- 172.21.15.16:0/2779295643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a00c3390 msgr2=0x7fb7a00ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.473+0000 7fb7a5c00640 1 --2- 172.21.15.16:0/2779295643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a00c3390 0x7fb7a00ca850 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00644c0 tx=0x7fb78c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.473+0000 7fb7a5c00640 1 -- 172.21.15.16:0/2779295643 shutdown_connections 2024-08-17T02:46:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.473+0000 7fb7a5c00640 1 --2- 172.21.15.16:0/2779295643 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7a0088f20 0x7fb7a00cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.473+0000 7fb7a5c00640 1 --2- 172.21.15.16:0/2779295643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a00c3390 0x7fb7a00ca850 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.473+0000 7fb7a5c00640 1 --2- 172.21.15.16:0/2779295643 >> v2:172.21.15.142:3300/0 conn(0x7fb7a0158e20 0x7fb7a00ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.473+0000 7fb7a5c00640 1 -- 172.21.15.16:0/2779295643 >> 172.21.15.16:0/2779295643 conn(0x7fb7a00cfd70 msgr2=0x7fb7a00d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.473+0000 7fb7a5c00640 1 -- 172.21.15.16:0/2779295643 shutdown_connections 2024-08-17T02:46:26.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.473+0000 7fb7a5c00640 1 -- 172.21.15.16:0/2779295643 wait complete. 2024-08-17T02:46:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.474+0000 7fb7a5c00640 1 Processor -- start 2024-08-17T02:46:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.475+0000 7fb7a5c00640 1 -- start start 2024-08-17T02:46:26.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.475+0000 7fb7a5c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0088f20 0x7fb7a00c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.475+0000 7fb7a5c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a00c3390 0x7fb7a0168560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.475+0000 7fb7a5c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7a0158e20 0x7fb7a015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.475+0000 7fb7a5c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a00c50f0 con 0x7fb7a0088f20 2024-08-17T02:46:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.475+0000 7fb7a5c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a00c53d0 con 0x7fb7a00c3390 2024-08-17T02:46:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.475+0000 7fb7a5c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a00c56b0 con 0x7fb7a0158e20 2024-08-17T02:46:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.476+0000 7fb79ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0088f20 0x7fb7a00c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.476+0000 7fb79ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0088f20 0x7fb7a00c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34376/0 (socket says 172.21.15.16:34376) 2024-08-17T02:46:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.476+0000 7fb79ea00640 1 -- 172.21.15.16:0/3253501711 learned_addr learned my addr 172.21.15.16:0/3253501711 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.476+0000 7fb79e000640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a00c3390 0x7fb7a0168560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.476+0000 7fb79f400640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7a0158e20 0x7fb7a015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.476+0000 7fb79ea00640 1 -- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7a0158e20 msgr2=0x7fb7a015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.476+0000 7fb79ea00640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7a0158e20 0x7fb7a015cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.476+0000 7fb79ea00640 1 -- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a00c3390 msgr2=0x7fb7a0168560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.476+0000 7fb79ea00640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a00c3390 0x7fb7a0168560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.476+0000 7fb79ea00640 1 -- 172.21.15.16:0/3253501711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c009450 con 0x7fb7a0088f20 2024-08-17T02:46:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.477+0000 7fb79ea00640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0088f20 0x7fb7a00c4750 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb79000d890 tx=0x7fb790008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.477+0000 7fb796000640 1 -- 172.21.15.16:0/3253501711 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79000de20 con 0x7fb7a0088f20 2024-08-17T02:46:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.477+0000 7fb796000640 1 -- 172.21.15.16:0/3253501711 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7900098b0 con 0x7fb7a0088f20 2024-08-17T02:46:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.477+0000 7fb7a5c00640 1 -- 172.21.15.16:0/3253501711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a00c5990 con 0x7fb7a0088f20 2024-08-17T02:46:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.477+0000 7fb796000640 1 -- 172.21.15.16:0/3253501711 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790009c50 con 0x7fb7a0088f20 2024-08-17T02:46:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.477+0000 7fb7a5c00640 1 -- 172.21.15.16:0/3253501711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a00b3340 con 0x7fb7a0088f20 2024-08-17T02:46:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.478+0000 7fb7a5c00640 1 -- 172.21.15.16:0/3253501711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a00c5e10 con 0x7fb7a0088f20 2024-08-17T02:46:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.479+0000 7fb796000640 1 -- 172.21.15.16:0/3253501711 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fb790014620 con 0x7fb7a0088f20 2024-08-17T02:46:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.480+0000 7fb796000640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb780065150 0x7fb780067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.480+0000 7fb796000640 1 -- 172.21.15.16:0/3253501711 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb790082bf0 con 0x7fb7a0088f20 2024-08-17T02:46:26.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.480+0000 7fb79e000640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb780065150 0x7fb780067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.481+0000 7fb79e000640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb780065150 0x7fb780067600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb78c034000 tx=0x7fb78c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:26.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.484+0000 7fb796000640 1 -- 172.21.15.16:0/3253501711 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb790050cc0 con 0x7fb7a0088f20 2024-08-17T02:46:26.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.665+0000 7fb7a5c00640 1 -- 172.21.15.16:0/3253501711 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb7a0071050 con 0x7fb7a0088f20 2024-08-17T02:46:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.666+0000 7fb796000640 1 -- 172.21.15.16:0/3253501711 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb7900542e0 con 0x7fb7a0088f20 2024-08-17T02:46:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.667+0000 7fb7a5c00640 1 -- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb780065150 msgr2=0x7fb780067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.667+0000 7fb7a5c00640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb780065150 0x7fb780067600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb78c034000 tx=0x7fb78c032000 comp rx=0 tx=0).stop 2024-08-17T02:46:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.667+0000 7fb7a5c00640 1 -- 172.21.15.16:0/3253501711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0088f20 msgr2=0x7fb7a00c4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.667+0000 7fb7a5c00640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0088f20 0x7fb7a00c4750 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb79000d890 tx=0x7fb790008040 comp rx=0 tx=0).stop 2024-08-17T02:46:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.667+0000 7fb7a5c00640 1 -- 172.21.15.16:0/3253501711 shutdown_connections 2024-08-17T02:46:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.667+0000 7fb7a5c00640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb780065150 0x7fb780067600 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.667+0000 7fb7a5c00640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb7a0158e20 0x7fb7a015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.667+0000 7fb7a5c00640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7a00c3390 0x7fb7a0168560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.667+0000 7fb7a5c00640 1 --2- 172.21.15.16:0/3253501711 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0088f20 0x7fb7a00c4750 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.668+0000 7fb7a5c00640 1 -- 172.21.15.16:0/3253501711 >> 172.21.15.16:0/3253501711 conn(0x7fb7a00cfd70 msgr2=0x7fb7a00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.668+0000 7fb7a5c00640 1 -- 172.21.15.16:0/3253501711 shutdown_connections 2024-08-17T02:46:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.668+0000 7fb7a5c00640 1 -- 172.21.15.16:0/3253501711 wait complete. 2024-08-17T02:46:26.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000521811 2024-08-17T02:46:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.836+0000 7f8d97200640 1 Processor -- start 2024-08-17T02:46:26.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.836+0000 7f8d97200640 1 -- start start 2024-08-17T02:46:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d97200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d9806d7a0 0x7f8d9806db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d97200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d980ae6b0 0x7f8d98154d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d97200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d980af860 con 0x7f8d980c8060 2024-08-17T02:46:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d97200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d980aec50 con 0x7f8d980ae6b0 2024-08-17T02:46:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d97200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d980aef30 con 0x7f8d9806d7a0 2024-08-17T02:46:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d96800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d980ae6b0 0x7f8d98154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d96800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d980ae6b0 0x7f8d98154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34378/0 (socket says 172.21.15.16:34378) 2024-08-17T02:46:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d96800640 1 -- 172.21.15.16:0/3998850332 learned_addr learned my addr 172.21.15.16:0/3998850332 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d95e00640 1 --2- 172.21.15.16:0/3998850332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d9806d7a0 0x7f8d9806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d96800640 1 -- 172.21.15.16:0/3998850332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d9806d7a0 msgr2=0x7f8d9806db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d96800640 1 --2- 172.21.15.16:0/3998850332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d9806d7a0 0x7f8d9806db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d96800640 1 -- 172.21.15.16:0/3998850332 >> v1:172.21.15.16:6789/0 conn(0x7f8d980c8060 legacy=0x7f8d980caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d96800640 1 -- 172.21.15.16:0/3998850332 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d980af210 con 0x7f8d980ae6b0 2024-08-17T02:46:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.838+0000 7f8d95e00640 1 --2- 172.21.15.16:0/3998850332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d9806d7a0 0x7f8d9806db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.839+0000 7f8d96800640 1 --2- 172.21.15.16:0/3998850332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d980ae6b0 0x7f8d98154d40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000d1a0 tx=0x7f8d9000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea9706f1184f66e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.840+0000 7f8d95400640 1 -- 172.21.15.16:0/3998850332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90033530 con 0x7f8d980ae6b0 2024-08-17T02:46:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.840+0000 7f8d95400640 1 -- 172.21.15.16:0/3998850332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d90033690 con 0x7f8d980ae6b0 2024-08-17T02:46:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.840+0000 7f8d95400640 1 -- 172.21.15.16:0/3998850332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90033a70 con 0x7f8d980ae6b0 2024-08-17T02:46:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.840+0000 7f8d97200640 1 -- 172.21.15.16:0/3998850332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d980ae6b0 msgr2=0x7f8d98154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.840+0000 7f8d97200640 1 --2- 172.21.15.16:0/3998850332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d980ae6b0 0x7f8d98154d40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000d1a0 tx=0x7f8d9000ba50 comp rx=0 tx=0).stop 2024-08-17T02:46:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.840+0000 7f8d97200640 1 -- 172.21.15.16:0/3998850332 shutdown_connections 2024-08-17T02:46:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.840+0000 7f8d97200640 1 --2- 172.21.15.16:0/3998850332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d980ae6b0 0x7f8d98154d40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.840+0000 7f8d97200640 1 --2- 172.21.15.16:0/3998850332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d9806d7a0 0x7f8d9806db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.840+0000 7f8d97200640 1 -- 172.21.15.16:0/3998850332 >> 172.21.15.16:0/3998850332 conn(0x7f8d98067750 msgr2=0x7f8d980cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.840+0000 7f8d97200640 1 -- 172.21.15.16:0/3998850332 shutdown_connections 2024-08-17T02:46:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.840+0000 7f8d97200640 1 -- 172.21.15.16:0/3998850332 wait complete. 2024-08-17T02:46:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.842+0000 7f8d97200640 1 Processor -- start 2024-08-17T02:46:26.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.842+0000 7f8d97200640 1 -- start start 2024-08-17T02:46:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d97200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d9806d7a0 0x7f8d980c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d97200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d980ae6b0 0x7f8d98151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d97200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d980c8060 0x7f8d980c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d97200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9816c4f0 con 0x7f8d980ae6b0 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d97200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9816c740 con 0x7f8d9806d7a0 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d97200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9816ca20 con 0x7f8d980c8060 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d980ae6b0 0x7f8d98151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d8fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d980ae6b0 0x7f8d98151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34392/0 (socket says 172.21.15.16:34392) 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d8fe00640 1 -- 172.21.15.16:0/3449017647 learned_addr learned my addr 172.21.15.16:0/3449017647 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d8fe00640 1 -- 172.21.15.16:0/3449017647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d980c8060 msgr2=0x7f8d980c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d8fe00640 1 --2- 172.21.15.16:0/3449017647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d980c8060 0x7f8d980c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d8fe00640 1 -- 172.21.15.16:0/3449017647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d9806d7a0 msgr2=0x7f8d980c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d8fe00640 1 --2- 172.21.15.16:0/3449017647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d9806d7a0 0x7f8d980c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.843+0000 7f8d8fe00640 1 -- 172.21.15.16:0/3449017647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9000bcf0 con 0x7f8d980ae6b0 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.844+0000 7f8d8fe00640 1 --2- 172.21.15.16:0/3449017647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d980ae6b0 0x7f8d98151f40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000c4c0 tx=0x7f8d80007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.844+0000 7f8d8d600640 1 -- 172.21.15.16:0/3449017647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80011590 con 0x7f8d980ae6b0 2024-08-17T02:46:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.844+0000 7f8d8d600640 1 -- 172.21.15.16:0/3449017647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d80011710 con 0x7f8d980ae6b0 2024-08-17T02:46:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.844+0000 7f8d8d600640 1 -- 172.21.15.16:0/3449017647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80011ad0 con 0x7f8d980ae6b0 2024-08-17T02:46:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.844+0000 7f8d97200640 1 -- 172.21.15.16:0/3449017647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9816cd00 con 0x7f8d980ae6b0 2024-08-17T02:46:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.844+0000 7f8d97200640 1 -- 172.21.15.16:0/3449017647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9816d350 con 0x7f8d980ae6b0 2024-08-17T02:46:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.845+0000 7f8d97200640 1 -- 172.21.15.16:0/3449017647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9816d180 con 0x7f8d980ae6b0 2024-08-17T02:46:26.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.846+0000 7f8d8d600640 1 -- 172.21.15.16:0/3449017647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f8d8001b3b0 con 0x7f8d980ae6b0 2024-08-17T02:46:26.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.847+0000 7f8d8d600640 1 --2- 172.21.15.16:0/3449017647 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8d74065150 0x7f8d74067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:26.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.847+0000 7f8d8d600640 1 -- 172.21.15.16:0/3449017647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d800819b0 con 0x7f8d980ae6b0 2024-08-17T02:46:26.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.847+0000 7f8d95e00640 1 --2- 172.21.15.16:0/3449017647 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8d74065150 0x7f8d74067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:26.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.848+0000 7f8d95e00640 1 --2- 172.21.15.16:0/3449017647 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8d74065150 0x7f8d74067600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8d88006e70 tx=0x7f8d88008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:26.851+0000 7f8d8d600640 1 -- 172.21.15.16:0/3449017647 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f8d8004f9d0 con 0x7f8d980ae6b0 2024-08-17T02:46:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:27.059+0000 7f8d97200640 1 -- 172.21.15.16:0/3449017647 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000521811", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d98071050 con 0x7f8d74065150 2024-08-17T02:46:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.950+0000 7f8d8d600640 1 -- 172.21.15.16:0/3449017647 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f8d98071050 con 0x7f8d74065150 2024-08-17T02:46:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:Volume created successfully (no MDS daemons created) 2024-08-17T02:46:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.954+0000 7f8d97200640 1 -- 172.21.15.16:0/3449017647 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8d74065150 msgr2=0x7f8d74067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.954+0000 7f8d97200640 1 --2- 172.21.15.16:0/3449017647 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8d74065150 0x7f8d74067600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8d88006e70 tx=0x7f8d88008040 comp rx=0 tx=0).stop 2024-08-17T02:46:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.954+0000 7f8d97200640 1 -- 172.21.15.16:0/3449017647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d980ae6b0 msgr2=0x7f8d98151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.954+0000 7f8d97200640 1 --2- 172.21.15.16:0/3449017647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d980ae6b0 0x7f8d98151f40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000c4c0 tx=0x7f8d80007ef0 comp rx=0 tx=0).stop 2024-08-17T02:46:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.954+0000 7f8d97200640 1 -- 172.21.15.16:0/3449017647 shutdown_connections 2024-08-17T02:46:29.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.954+0000 7f8d97200640 1 --2- 172.21.15.16:0/3449017647 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8d74065150 0x7f8d74067600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.954+0000 7f8d97200640 1 --2- 172.21.15.16:0/3449017647 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f8d980c8060 0x7f8d980c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.954+0000 7f8d97200640 1 --2- 172.21.15.16:0/3449017647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d980ae6b0 0x7f8d98151f40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.954+0000 7f8d97200640 1 --2- 172.21.15.16:0/3449017647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d9806d7a0 0x7f8d980c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.954+0000 7f8d97200640 1 -- 172.21.15.16:0/3449017647 >> 172.21.15.16:0/3449017647 conn(0x7f8d98067750 msgr2=0x7f8d980ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.955+0000 7f8d97200640 1 -- 172.21.15.16:0/3449017647 shutdown_connections 2024-08-17T02:46:29.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:29.955+0000 7f8d97200640 1 -- 172.21.15.16:0/3449017647 wait complete. 2024-08-17T02:46:29.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-08-17T02:46:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.133+0000 7faac0600640 1 Processor -- start 2024-08-17T02:46:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.133+0000 7faac0600640 1 -- start start 2024-08-17T02:46:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.134+0000 7faac0600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faab8140580 0x7faab80c9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.134+0000 7faac0600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab80caa80 con 0x7faab80b1440 2024-08-17T02:46:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.134+0000 7faac0600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab80c9e70 con 0x7faab81401b0 2024-08-17T02:46:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.134+0000 7faac0600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab80c9c90 con 0x7faab8140580 2024-08-17T02:46:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.135+0000 7faabe600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faab8140580 0x7faab80c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.135+0000 7faabe600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faab8140580 0x7faab80c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34398/0 (socket says 172.21.15.16:34398) 2024-08-17T02:46:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.135+0000 7faabe600640 1 -- 172.21.15.16:0/2611269234 learned_addr learned my addr 172.21.15.16:0/2611269234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:30.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.135+0000 7faabd200640 1 --1- 172.21.15.16:0/2611269234 >> v1:172.21.15.16:6789/0 conn(0x7faab80b1440 0x7faab80c9040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54210/0 (socket says 172.21.15.16:54210) 2024-08-17T02:46:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.135+0000 7faabe600640 1 -- 172.21.15.16:0/2611269234 >> v1:172.21.15.142:6789/0 conn(0x7faab81401b0 legacy=0x7faab80b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.135+0000 7faabe600640 1 -- 172.21.15.16:0/2611269234 >> v1:172.21.15.16:6789/0 conn(0x7faab80b1440 legacy=0x7faab80c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.135+0000 7faabe600640 1 -- 172.21.15.16:0/2611269234 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab80ca150 con 0x7faab8140580 2024-08-17T02:46:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.135+0000 7faab7e00640 1 -- 172.21.15.16:0/2611269234 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397939780 0 0) 0x7faab80caa80 con 0x7faab80b1440 2024-08-17T02:46:30.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.136+0000 7faabe600640 1 --2- 172.21.15.16:0/2611269234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8140580 0x7faab80c9750 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faab0009f20 tx=0x7faab0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be9e32b40dd588c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:30.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.136+0000 7faab7e00640 1 -- 172.21.15.16:0/2611269234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0032470 con 0x7faab8140580 2024-08-17T02:46:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.136+0000 7faab7e00640 1 -- 172.21.15.16:0/2611269234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab00325d0 con 0x7faab8140580 2024-08-17T02:46:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.136+0000 7faab7e00640 1 -- 172.21.15.16:0/2611269234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab0032970 con 0x7faab8140580 2024-08-17T02:46:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.137+0000 7faac0600640 1 -- 172.21.15.16:0/2611269234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8140580 msgr2=0x7faab80c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.137+0000 7faac0600640 1 --2- 172.21.15.16:0/2611269234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8140580 0x7faab80c9750 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faab0009f20 tx=0x7faab0009840 comp rx=0 tx=0).stop 2024-08-17T02:46:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.137+0000 7faac0600640 1 -- 172.21.15.16:0/2611269234 shutdown_connections 2024-08-17T02:46:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.137+0000 7faac0600640 1 --2- 172.21.15.16:0/2611269234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab8140580 0x7faab80c9750 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.137+0000 7faac0600640 1 -- 172.21.15.16:0/2611269234 >> 172.21.15.16:0/2611269234 conn(0x7faab80cb9b0 msgr2=0x7faab80cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.137+0000 7faac0600640 1 -- 172.21.15.16:0/2611269234 shutdown_connections 2024-08-17T02:46:30.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.137+0000 7faac0600640 1 -- 172.21.15.16:0/2611269234 wait complete. 2024-08-17T02:46:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.139+0000 7faac0600640 1 Processor -- start 2024-08-17T02:46:30.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.139+0000 7faac0600640 1 -- start start 2024-08-17T02:46:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faac0600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab80b1440 0x7faab8168100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faac0600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab81401b0 0x7faab814d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faac0600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab8140580 0x7faab814c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faac0600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab814c3d0 con 0x7faab81401b0 2024-08-17T02:46:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faac0600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab80ad070 con 0x7faab8140580 2024-08-17T02:46:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faac0600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab814b7c0 con 0x7faab80b1440 2024-08-17T02:46:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faabd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab81401b0 0x7faab814d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faabd200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab81401b0 0x7faab814d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34404/0 (socket says 172.21.15.16:34404) 2024-08-17T02:46:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faabd200640 1 -- 172.21.15.16:0/2922544184 learned_addr learned my addr 172.21.15.16:0/2922544184 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faabe600640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab8140580 0x7faab814c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faabdc00640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab80b1440 0x7faab8168100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faabd200640 1 -- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab80b1440 msgr2=0x7faab8168100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faabd200640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab80b1440 0x7faab8168100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faabd200640 1 -- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab8140580 msgr2=0x7faab814c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faabd200640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab8140580 0x7faab814c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faabd200640 1 -- 172.21.15.16:0/2922544184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab0009ae0 con 0x7faab81401b0 2024-08-17T02:46:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.140+0000 7faabe600640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab8140580 0x7faab814c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:30.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.141+0000 7faabd200640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab81401b0 0x7faab814d740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faaa400c5e0 tx=0x7faaa4007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.141+0000 7faab4c00640 1 -- 172.21.15.16:0/2922544184 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4007300 con 0x7faab81401b0 2024-08-17T02:46:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.141+0000 7faab4c00640 1 -- 172.21.15.16:0/2922544184 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa4007480 con 0x7faab81401b0 2024-08-17T02:46:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.141+0000 7faab4c00640 1 -- 172.21.15.16:0/2922544184 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40049f0 con 0x7faab81401b0 2024-08-17T02:46:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.141+0000 7faac0600640 1 -- 172.21.15.16:0/2922544184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab814baa0 con 0x7faab81401b0 2024-08-17T02:46:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.141+0000 7faac0600640 1 -- 172.21.15.16:0/2922544184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab814c060 con 0x7faab81401b0 2024-08-17T02:46:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.142+0000 7faac0600640 1 -- 172.21.15.16:0/2922544184 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab814c540 con 0x7faab81401b0 2024-08-17T02:46:30.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.143+0000 7faab4c00640 1 -- 172.21.15.16:0/2922544184 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7faaa40113a0 con 0x7faab81401b0 2024-08-17T02:46:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.144+0000 7faab4c00640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7faa98065150 0x7faa98067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.144+0000 7faab4c00640 1 -- 172.21.15.16:0/2922544184 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6603+0+0 (secure 0 0 0) 0x7faaa4081960 con 0x7faab81401b0 2024-08-17T02:46:30.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.145+0000 7faabdc00640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7faa98065150 0x7faa98067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.145+0000 7faabdc00640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7faa98065150 0x7faa98067600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faaac0037c0 tx=0x7faaac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.148+0000 7faab4c00640 1 -- 172.21.15.16:0/2922544184 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7faaa404f680 con 0x7faab81401b0 2024-08-17T02:46:30.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.357+0000 7faac0600640 1 -- 172.21.15.16:0/2922544184 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7faab8064db0 con 0x7faa98065150 2024-08-17T02:46:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.359+0000 7faab4c00640 1 -- 172.21.15.16:0/2922544184 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7faab8064db0 con 0x7faa98065150 2024-08-17T02:46:30.360 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-08-17T02:46:30.360 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:30.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephfs" 2024-08-17T02:46:30.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-08-17T02:46:30.360 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-08-17T02:46:30.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volume_0000000000521811" 2024-08-17T02:46:30.361 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-08-17T02:46:30.361 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-08-17T02:46:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.361+0000 7faac0600640 1 -- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7faa98065150 msgr2=0x7faa98067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.361+0000 7faac0600640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7faa98065150 0x7faa98067600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faaac0037c0 tx=0x7faaac009040 comp rx=0 tx=0).stop 2024-08-17T02:46:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.361+0000 7faac0600640 1 -- 172.21.15.16:0/2922544184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab81401b0 msgr2=0x7faab814d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.361+0000 7faac0600640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab81401b0 0x7faab814d740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7faaa400c5e0 tx=0x7faaa4007d00 comp rx=0 tx=0).stop 2024-08-17T02:46:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.361+0000 7faac0600640 1 -- 172.21.15.16:0/2922544184 shutdown_connections 2024-08-17T02:46:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.361+0000 7faac0600640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7faa98065150 0x7faa98067600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.361+0000 7faac0600640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faab8140580 0x7faab814c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.361+0000 7faac0600640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faab81401b0 0x7faab814d740 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.361+0000 7faac0600640 1 --2- 172.21.15.16:0/2922544184 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7faab80b1440 0x7faab8168100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.362+0000 7faac0600640 1 -- 172.21.15.16:0/2922544184 >> 172.21.15.16:0/2922544184 conn(0x7faab80cb9b0 msgr2=0x7faab80c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:30.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.362+0000 7faac0600640 1 -- 172.21.15.16:0/2922544184 shutdown_connections 2024-08-17T02:46:30.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.362+0000 7faac0600640 1 -- 172.21.15.16:0/2922544184 wait complete. 2024-08-17T02:46:30.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool ls detail --format=json 2024-08-17T02:46:30.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.530+0000 7f0569800640 1 Processor -- start 2024-08-17T02:46:30.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.530+0000 7f0569800640 1 -- start start 2024-08-17T02:46:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.531+0000 7f0569800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0564156990 0x7f05640b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.531+0000 7f0569800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056415b030 con 0x7f05640adec0 2024-08-17T02:46:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.532+0000 7f0569800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05640cc2e0 con 0x7f0564156d60 2024-08-17T02:46:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.532+0000 7f0569800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05640b1110 con 0x7f0564156990 2024-08-17T02:46:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.532+0000 7f0563400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f05640adec0 0x7f05640ae940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54224/0 (socket says 172.21.15.16:54224) 2024-08-17T02:46:30.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.532+0000 7f0563400640 1 -- 172.21.15.16:0/4034162977 learned_addr learned my addr 172.21.15.16:0/4034162977 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:46:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.532+0000 7f0562a00640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0564156990 0x7f05640b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.532+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821157676 0 0) 0x7f056415b030 con 0x7f05640adec0 2024-08-17T02:46:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.532+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05480034e0 con 0x7f05640adec0 2024-08-17T02:46:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 679612583 0 0) 0x7f05480034e0 con 0x7f05640adec0 2024-08-17T02:46:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0564156990 msgr2=0x7f05640b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0564156990 0x7f05640b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 >> v1:172.21.15.142:6789/0 conn(0x7f0564156d60 legacy=0x7f05640ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056415a700 con 0x7f05640adec0 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7f055c0017f0 con 0x7f05640adec0 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 >> v1:172.21.15.16:6789/0 conn(0x7f05640adec0 legacy=0x7f05640ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0564156990 0x7f0548003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548003f40 0x7f0548004780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0548004cc0 0x7f0548005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05640b1110 con 0x7f0548003f40 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054800bab0 con 0x7f0548004cc0 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054800bc70 con 0x7f0564156990 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0562000640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548003f40 0x7f0548004780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0562a00640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0564156990 0x7f0548003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.533+0000 7f0563400640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0548004cc0 0x7f0548005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.534+0000 7f0562000640 1 -- 172.21.15.16:0/4034162977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0564156990 msgr2=0x7f0548003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.534+0000 7f0562000640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0564156990 0x7f0548003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.534+0000 7f0562000640 1 -- 172.21.15.16:0/4034162977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0548004cc0 msgr2=0x7f0548005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.534+0000 7f0562000640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0548004cc0 0x7f0548005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.534+0000 7f0562000640 1 -- 172.21.15.16:0/4034162977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05480034e0 con 0x7f0548003f40 2024-08-17T02:46:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.534+0000 7f0563400640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0548004cc0 0x7f0548005130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.535+0000 7f0562000640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548003f40 0x7f0548004780 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0550001050 tx=0x7f0550007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fc9a0a87d891d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.535+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055002aa60 con 0x7f0548003f40 2024-08-17T02:46:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.535+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055002abe0 con 0x7f0548003f40 2024-08-17T02:46:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.535+0000 7f0561600640 1 -- 172.21.15.16:0/4034162977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05500323f0 con 0x7f0548003f40 2024-08-17T02:46:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.535+0000 7f0569800640 1 -- 172.21.15.16:0/4034162977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548003f40 msgr2=0x7f0548004780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.535+0000 7f0569800640 1 --2- 172.21.15.16:0/4034162977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0548003f40 0x7f0548004780 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0550001050 tx=0x7f0550007ca0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.535+0000 7f0563400640 1 -- 172.21.15.16:0/4034162977 reap_dead start 2024-08-17T02:46:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.535+0000 7f0569800640 1 -- 172.21.15.16:0/4034162977 shutdown_connections 2024-08-17T02:46:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.535+0000 7f0569800640 1 -- 172.21.15.16:0/4034162977 >> 172.21.15.16:0/4034162977 conn(0x7f0564067750 msgr2=0x7f05640cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.535+0000 7f0569800640 1 -- 172.21.15.16:0/4034162977 shutdown_connections 2024-08-17T02:46:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.535+0000 7f0569800640 1 -- 172.21.15.16:0/4034162977 wait complete. 2024-08-17T02:46:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.537+0000 7f0569800640 1 Processor -- start 2024-08-17T02:46:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.537+0000 7f0569800640 1 -- start start 2024-08-17T02:46:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0569800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548003f40 0x7f05640ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0569800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0564157130 0x7f0564158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0569800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0564151810 0x7f05640c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0569800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0564158400 con 0x7f0564157130 2024-08-17T02:46:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0569800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05641575a0 con 0x7f0564151810 2024-08-17T02:46:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0569800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05640c78b0 con 0x7f0548003f40 2024-08-17T02:46:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0562000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0564157130 0x7f0564158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0562000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0564157130 0x7f0564158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34414/0 (socket says 172.21.15.16:34414) 2024-08-17T02:46:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0562000640 1 -- 172.21.15.16:0/2101348921 learned_addr learned my addr 172.21.15.16:0/2101348921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0563400640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0564151810 0x7f05640c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0562000640 1 -- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548003f40 msgr2=0x7f05640ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0562a00640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548003f40 0x7f05640ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0562000640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548003f40 0x7f05640ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0562000640 1 -- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0564151810 msgr2=0x7f05640c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0562000640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0564151810 0x7f05640c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.538+0000 7f0562000640 1 -- 172.21.15.16:0/2101348921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05641577f0 con 0x7f0564157130 2024-08-17T02:46:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.539+0000 7f0563400640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0564151810 0x7f05640c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:46:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.539+0000 7f0562a00640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548003f40 0x7f05640ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:46:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.539+0000 7f0562000640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0564157130 0x7f0564158890 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f055002a730 tx=0x7f0550003000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.539+0000 7f055a000640 1 -- 172.21.15.16:0/2101348921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550033dc0 con 0x7f0564157130 2024-08-17T02:46:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.539+0000 7f055a000640 1 -- 172.21.15.16:0/2101348921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0550007050 con 0x7f0564157130 2024-08-17T02:46:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.539+0000 7f055a000640 1 -- 172.21.15.16:0/2101348921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550007410 con 0x7f0564157130 2024-08-17T02:46:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.539+0000 7f0569800640 1 -- 172.21.15.16:0/2101348921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0564157ad0 con 0x7f0564157130 2024-08-17T02:46:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.539+0000 7f0569800640 1 -- 172.21.15.16:0/2101348921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0564158090 con 0x7f0564157130 2024-08-17T02:46:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.540+0000 7f0569800640 1 -- 172.21.15.16:0/2101348921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0564071050 con 0x7f0564157130 2024-08-17T02:46:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.541+0000 7f055a000640 1 -- 172.21.15.16:0/2101348921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f055002ad50 con 0x7f0564157130 2024-08-17T02:46:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.542+0000 7f055a000640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0544065150 0x7f0544067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.542+0000 7f055a000640 1 -- 172.21.15.16:0/2101348921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f05500a2990 con 0x7f0564157130 2024-08-17T02:46:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.543+0000 7f0562a00640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0544065150 0x7f0544067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.543+0000 7f0562a00640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0544065150 0x7f0544067600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0554009360 tx=0x7f0554008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.546+0000 7f055a000640 1 -- 172.21.15.16:0/2101348921 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f05500706f0 con 0x7f0564157130 2024-08-17T02:46:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.727+0000 7f0569800640 1 -- 172.21.15.16:0/2101348921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool ls", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f0564071050 con 0x7f0564157130 2024-08-17T02:46:30.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.727+0000 7f055a000640 1 -- 172.21.15.16:0/2101348921 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls", "detail": "detail", "format": "json"}]=0 v36) v1 ==== 97+0+8050 (secure 0 0 0) 0x7f0550073d10 con 0x7f0564157130 2024-08-17T02:46:30.729 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:46:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.729+0000 7f0569800640 1 -- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0544065150 msgr2=0x7f0544067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.729+0000 7f0569800640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0544065150 0x7f0544067600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0554009360 tx=0x7f0554008040 comp rx=0 tx=0).stop 2024-08-17T02:46:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.729+0000 7f0569800640 1 -- 172.21.15.16:0/2101348921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0564157130 msgr2=0x7f0564158890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.729+0000 7f0569800640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0564157130 0x7f0564158890 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f055002a730 tx=0x7f0550003000 comp rx=0 tx=0).stop 2024-08-17T02:46:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.729+0000 7f0569800640 1 -- 172.21.15.16:0/2101348921 shutdown_connections 2024-08-17T02:46:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.729+0000 7f0569800640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0544065150 0x7f0544067600 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.729+0000 7f0569800640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0564151810 0x7f05640c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.729+0000 7f0569800640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0564157130 0x7f0564158890 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.729+0000 7f0569800640 1 --2- 172.21.15.16:0/2101348921 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0548003f40 0x7f05640ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.729+0000 7f0569800640 1 -- 172.21.15.16:0/2101348921 >> 172.21.15.16:0/2101348921 conn(0x7f0564067750 msgr2=0x7f0564152dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.730+0000 7f0569800640 1 -- 172.21.15.16:0/2101348921 shutdown_connections 2024-08-17T02:46:30.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.730+0000 7f0569800640 1 -- 172.21.15.16:0/2101348921 wait complete. 2024-08-17T02:46:30.746 INFO:teuthology.orchestra.run.smithi016.stdout:[{"pool_id":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool_id":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:46:14.322032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool_id":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:46:15.170652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool_id":6,"pool_name":"cephfs.volume_0000000000521811.meta","create_time":"2024-08-17T02:46:27.102528+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000521811"}}},{"pool_id":7,"pool_name":"cephfs.volume_0000000000521811.data","create_time":"2024-08-17T02:46:27.975177+0000","flags":163841,"flags_names":"hashpspool,creating,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000521811"}}}] 2024-08-17T02:46:30.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs get volume_0000000000521811 --format=json 2024-08-17T02:46:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.897+0000 7f9c60400640 1 Processor -- start 2024-08-17T02:46:30.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.897+0000 7f9c60400640 1 -- start start 2024-08-17T02:46:30.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.898+0000 7f9c60400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c581401b0 0x7f9c580caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.898+0000 7f9c60400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9c58140580 0x7f9c580b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.898+0000 7f9c60400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c580b1270 0x7f9c580c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.898+0000 7f9c60400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c580cb450 con 0x7f9c581401b0 2024-08-17T02:46:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.898+0000 7f9c60400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c580b16d0 con 0x7f9c58140580 2024-08-17T02:46:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.898+0000 7f9c60400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c580caa80 con 0x7f9c580b1270 2024-08-17T02:46:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.898+0000 7f9c5da00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c581401b0 0x7f9c580caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.899+0000 7f9c5da00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c581401b0 0x7f9c580caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34418/0 (socket says 172.21.15.16:34418) 2024-08-17T02:46:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.899+0000 7f9c5da00640 1 -- 172.21.15.16:0/3089201327 learned_addr learned my addr 172.21.15.16:0/3089201327 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.899+0000 7f9c5d000640 1 --2- 172.21.15.16:0/3089201327 >> v2:172.21.15.142:3300/0 conn(0x7f9c58140580 0x7f9c580b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.899+0000 7f9c5e400640 1 --2- 172.21.15.16:0/3089201327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c580b1270 0x7f9c580c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.899+0000 7f9c5da00640 1 -- 172.21.15.16:0/3089201327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c580b1270 msgr2=0x7f9c580c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.899+0000 7f9c5da00640 1 --2- 172.21.15.16:0/3089201327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c580b1270 0x7f9c580c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.899+0000 7f9c5da00640 1 -- 172.21.15.16:0/3089201327 >> v2:172.21.15.142:3300/0 conn(0x7f9c58140580 msgr2=0x7f9c580b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:30.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.899+0000 7f9c5da00640 1 --2- 172.21.15.16:0/3089201327 >> v2:172.21.15.142:3300/0 conn(0x7f9c58140580 0x7f9c580b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.899+0000 7f9c5da00640 1 -- 172.21.15.16:0/3089201327 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c580c9e70 con 0x7f9c581401b0 2024-08-17T02:46:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.900+0000 7f9c5da00640 1 --2- 172.21.15.16:0/3089201327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c581401b0 0x7f9c580caf10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9c5806aa80 tx=0x7f9c4c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436e6f5480e5b7b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.900+0000 7f9c57e00640 1 -- 172.21.15.16:0/3089201327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c02ad40 con 0x7f9c581401b0 2024-08-17T02:46:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.900+0000 7f9c57e00640 1 -- 172.21.15.16:0/3089201327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c032440 con 0x7f9c581401b0 2024-08-17T02:46:30.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.900+0000 7f9c57e00640 1 -- 172.21.15.16:0/3089201327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c02aea0 con 0x7f9c581401b0 2024-08-17T02:46:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.901+0000 7f9c60400640 1 -- 172.21.15.16:0/3089201327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c581401b0 msgr2=0x7f9c580caf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.901+0000 7f9c60400640 1 --2- 172.21.15.16:0/3089201327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c581401b0 0x7f9c580caf10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9c5806aa80 tx=0x7f9c4c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.901+0000 7f9c60400640 1 -- 172.21.15.16:0/3089201327 shutdown_connections 2024-08-17T02:46:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.901+0000 7f9c60400640 1 --2- 172.21.15.16:0/3089201327 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c580b1270 0x7f9c580c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.901+0000 7f9c60400640 1 --2- 172.21.15.16:0/3089201327 >> v2:172.21.15.142:3300/0 conn(0x7f9c58140580 0x7f9c580b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.901+0000 7f9c60400640 1 --2- 172.21.15.16:0/3089201327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c581401b0 0x7f9c580caf10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.901+0000 7f9c60400640 1 -- 172.21.15.16:0/3089201327 >> 172.21.15.16:0/3089201327 conn(0x7f9c580cb9b0 msgr2=0x7f9c580cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.901+0000 7f9c60400640 1 -- 172.21.15.16:0/3089201327 shutdown_connections 2024-08-17T02:46:30.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.901+0000 7f9c60400640 1 -- 172.21.15.16:0/3089201327 wait complete. 2024-08-17T02:46:30.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.902+0000 7f9c60400640 1 Processor -- start 2024-08-17T02:46:30.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.903+0000 7f9c60400640 1 -- start start 2024-08-17T02:46:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.903+0000 7f9c60400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c580b1270 0x7f9c5815c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.903+0000 7f9c60400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c581401b0 0x7f9c5813d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.903+0000 7f9c60400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c58140580 0x7f9c5813a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c60400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5815d810 con 0x7f9c58140580 2024-08-17T02:46:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c60400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5815dca0 con 0x7f9c581401b0 2024-08-17T02:46:30.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c60400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5815cc00 con 0x7f9c580b1270 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c5e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c58140580 0x7f9c5813a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c5e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c58140580 0x7f9c5813a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34424/0 (socket says 172.21.15.16:34424) 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c5e400640 1 -- 172.21.15.16:0/3653944841 learned_addr learned my addr 172.21.15.16:0/3653944841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c5da00640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c580b1270 0x7f9c5815c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c5d000640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c581401b0 0x7f9c5813d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c5e400640 1 -- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c580b1270 msgr2=0x7f9c5815c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c5e400640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c580b1270 0x7f9c5815c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c5e400640 1 -- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c581401b0 msgr2=0x7f9c5813d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c5e400640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c581401b0 0x7f9c5813d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.904+0000 7f9c5e400640 1 -- 172.21.15.16:0/3653944841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c009450 con 0x7f9c58140580 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.905+0000 7f9c5e400640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c58140580 0x7f9c5813a310 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009df0 tx=0x7f9c5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.905+0000 7f9c54c00640 1 -- 172.21.15.16:0/3653944841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5000cd40 con 0x7f9c58140580 2024-08-17T02:46:30.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.905+0000 7f9c54c00640 1 -- 172.21.15.16:0/3653944841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5000f040 con 0x7f9c58140580 2024-08-17T02:46:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.905+0000 7f9c54c00640 1 -- 172.21.15.16:0/3653944841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c500146d0 con 0x7f9c58140580 2024-08-17T02:46:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.905+0000 7f9c60400640 1 -- 172.21.15.16:0/3653944841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5815cee0 con 0x7f9c58140580 2024-08-17T02:46:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.905+0000 7f9c60400640 1 -- 172.21.15.16:0/3653944841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5815d4a0 con 0x7f9c58140580 2024-08-17T02:46:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.906+0000 7f9c60400640 1 -- 172.21.15.16:0/3653944841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c58064db0 con 0x7f9c58140580 2024-08-17T02:46:30.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.907+0000 7f9c54c00640 1 -- 172.21.15.16:0/3653944841 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f9c50014a80 con 0x7f9c58140580 2024-08-17T02:46:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.908+0000 7f9c54c00640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9c38065150 0x7f9c38067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.908+0000 7f9c54c00640 1 -- 172.21.15.16:0/3653944841 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f9c50083440 con 0x7f9c58140580 2024-08-17T02:46:30.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.908+0000 7f9c5da00640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9c38065150 0x7f9c38067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:30.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.909+0000 7f9c5da00640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9c38065150 0x7f9c38067600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c034000 tx=0x7f9c4c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:30.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:30.912+0000 7f9c54c00640 1 -- 172.21.15.16:0/3653944841 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f9c50051190 con 0x7f9c58140580 2024-08-17T02:46:31.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.150+0000 7f9c60400640 1 -- 172.21.15.16:0/3653944841 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs get", "fs_name": "volume_0000000000521811", "format": "json"} v 0) v1 -- 0x7f9c5815f550 con 0x7f9c58140580 2024-08-17T02:46:31.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.150+0000 7f9c54c00640 1 -- 172.21.15.16:0/3653944841 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs get", "fs_name": "volume_0000000000521811", "format": "json"}]=0 v22) v1 ==== 110+0+1844 (secure 0 0 0) 0x7f9c50018750 con 0x7f9c58140580 2024-08-17T02:46:31.151 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:46:31.151 INFO:teuthology.orchestra.run.smithi016.stdout:{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:28.938945+0000","modified":"2024-08-17T02:46:30.943869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6834/3115083427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3115083427},{"type":"v1","addr":"172.21.15.16:6834","nonce":3115083427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000521811","balancer":"","standby_count_wanted":1},"id":3} 2024-08-17T02:46:31.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.152+0000 7f9c60400640 1 -- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9c38065150 msgr2=0x7f9c38067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.152+0000 7f9c60400640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9c38065150 0x7f9c38067600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c034000 tx=0x7f9c4c032000 comp rx=0 tx=0).stop 2024-08-17T02:46:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.152+0000 7f9c60400640 1 -- 172.21.15.16:0/3653944841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c58140580 msgr2=0x7f9c5813a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.152+0000 7f9c60400640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c58140580 0x7f9c5813a310 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009df0 tx=0x7f9c5000b040 comp rx=0 tx=0).stop 2024-08-17T02:46:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.152+0000 7f9c60400640 1 -- 172.21.15.16:0/3653944841 shutdown_connections 2024-08-17T02:46:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.152+0000 7f9c60400640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f9c38065150 0x7f9c38067600 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.152+0000 7f9c60400640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c58140580 0x7f9c5813a310 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.152+0000 7f9c60400640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c581401b0 0x7f9c5813d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.152+0000 7f9c60400640 1 --2- 172.21.15.16:0/3653944841 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9c580b1270 0x7f9c5815c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.152+0000 7f9c60400640 1 -- 172.21.15.16:0/3653944841 >> 172.21.15.16:0/3653944841 conn(0x7f9c580cb9b0 msgr2=0x7f9c580aed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.152+0000 7f9c60400640 1 -- 172.21.15.16:0/3653944841 shutdown_connections 2024-08-17T02:46:31.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.152+0000 7f9c60400640 1 -- 172.21.15.16:0/3653944841 wait complete. 2024-08-17T02:46:31.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000521811 --yes-i-really-mean-it 2024-08-17T02:46:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.321+0000 7fcf4a600640 1 Processor -- start 2024-08-17T02:46:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.321+0000 7fcf4a600640 1 -- start start 2024-08-17T02:46:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.322+0000 7fcf4a600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf44156c50 0x7fcf440c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.322+0000 7fcf4a600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fcf440adcf0 0x7fcf440ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.322+0000 7fcf4a600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf440abb00 0x7fcf440c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.322+0000 7fcf4a600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf440ab130 con 0x7fcf440abb00 2024-08-17T02:46:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.322+0000 7fcf4a600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf440ae5a0 con 0x7fcf440adcf0 2024-08-17T02:46:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.322+0000 7fcf4a600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440ae2c0 con 0x7fcf44156c50 2024-08-17T02:46:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.322+0000 7fcf43e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf440abb00 0x7fcf440c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.322+0000 7fcf43e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf440abb00 0x7fcf440c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34428/0 (socket says 172.21.15.16:34428) 2024-08-17T02:46:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.322+0000 7fcf43e00640 1 -- 172.21.15.16:0/1407207386 learned_addr learned my addr 172.21.15.16:0/1407207386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.322+0000 7fcf42a00640 1 --2- 172.21.15.16:0/1407207386 >> v2:172.21.15.142:3300/0 conn(0x7fcf440adcf0 0x7fcf440ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.323+0000 7fcf43400640 1 --2- 172.21.15.16:0/1407207386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf44156c50 0x7fcf440c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.323+0000 7fcf43e00640 1 -- 172.21.15.16:0/1407207386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf44156c50 msgr2=0x7fcf440c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.323+0000 7fcf43e00640 1 --2- 172.21.15.16:0/1407207386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf44156c50 0x7fcf440c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.323+0000 7fcf43e00640 1 -- 172.21.15.16:0/1407207386 >> v2:172.21.15.142:3300/0 conn(0x7fcf440adcf0 msgr2=0x7fcf440ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:46:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.323+0000 7fcf43e00640 1 --2- 172.21.15.16:0/1407207386 >> v2:172.21.15.142:3300/0 conn(0x7fcf440adcf0 0x7fcf440ab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.323+0000 7fcf43e00640 1 -- 172.21.15.16:0/1407207386 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440aaae0 con 0x7fcf440abb00 2024-08-17T02:46:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.323+0000 7fcf43400640 1 --2- 172.21.15.16:0/1407207386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf44156c50 0x7fcf440c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:46:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.324+0000 7fcf43e00640 1 --2- 172.21.15.16:0/1407207386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf440abb00 0x7fcf440c8bd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcf4406bc10 tx=0x7fcf3c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98ab8a13e9720c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.324+0000 7fcf42000640 1 -- 172.21.15.16:0/1407207386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c031400 con 0x7fcf440abb00 2024-08-17T02:46:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.324+0000 7fcf42000640 1 -- 172.21.15.16:0/1407207386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c02d050 con 0x7fcf440abb00 2024-08-17T02:46:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.324+0000 7fcf42000640 1 -- 172.21.15.16:0/1407207386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c0317d0 con 0x7fcf440abb00 2024-08-17T02:46:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.325+0000 7fcf4a600640 1 -- 172.21.15.16:0/1407207386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf440abb00 msgr2=0x7fcf440c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.325+0000 7fcf4a600640 1 --2- 172.21.15.16:0/1407207386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf440abb00 0x7fcf440c8bd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcf4406bc10 tx=0x7fcf3c009840 comp rx=0 tx=0).stop 2024-08-17T02:46:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.325+0000 7fcf4a600640 1 -- 172.21.15.16:0/1407207386 shutdown_connections 2024-08-17T02:46:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.325+0000 7fcf4a600640 1 --2- 172.21.15.16:0/1407207386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf440abb00 0x7fcf440c8bd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.325+0000 7fcf4a600640 1 --2- 172.21.15.16:0/1407207386 >> v2:172.21.15.142:3300/0 conn(0x7fcf440adcf0 0x7fcf440ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.325+0000 7fcf4a600640 1 --2- 172.21.15.16:0/1407207386 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf44156c50 0x7fcf440c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.325+0000 7fcf4a600640 1 -- 172.21.15.16:0/1407207386 >> 172.21.15.16:0/1407207386 conn(0x7fcf44067750 msgr2=0x7fcf440c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:46:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.325+0000 7fcf4a600640 1 -- 172.21.15.16:0/1407207386 shutdown_connections 2024-08-17T02:46:31.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.325+0000 7fcf4a600640 1 -- 172.21.15.16:0/1407207386 wait complete. 2024-08-17T02:46:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.326+0000 7fcf4a600640 1 Processor -- start 2024-08-17T02:46:31.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.326+0000 7fcf4a600640 1 -- start start 2024-08-17T02:46:31.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.327+0000 7fcf4a600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf440abb00 0x7fcf44148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:31.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.327+0000 7fcf4a600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf440adcf0 0x7fcf44069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:31.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.327+0000 7fcf4a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf44156c50 0x7fcf44158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:31.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.327+0000 7fcf4a600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf441499e0 con 0x7fcf44156c50 2024-08-17T02:46:31.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.327+0000 7fcf4a600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf44149440 con 0x7fcf440abb00 2024-08-17T02:46:31.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.327+0000 7fcf4a600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf44169350 con 0x7fcf440adcf0 2024-08-17T02:46:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.328+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf44156c50 0x7fcf44158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.328+0000 7fcf43e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf44156c50 0x7fcf44158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34432/0 (socket says 172.21.15.16:34432) 2024-08-17T02:46:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.328+0000 7fcf43e00640 1 -- 172.21.15.16:0/2247295970 learned_addr learned my addr 172.21.15.16:0/2247295970 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:46:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.328+0000 7fcf42a00640 1 --2- 172.21.15.16:0/2247295970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf440adcf0 0x7fcf44069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.328+0000 7fcf43400640 1 --2- 172.21.15.16:0/2247295970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf440abb00 0x7fcf44148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.328+0000 7fcf43e00640 1 -- 172.21.15.16:0/2247295970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf440adcf0 msgr2=0x7fcf44069b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.328+0000 7fcf43e00640 1 --2- 172.21.15.16:0/2247295970 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf440adcf0 0x7fcf44069b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.328+0000 7fcf43e00640 1 -- 172.21.15.16:0/2247295970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf440abb00 msgr2=0x7fcf44148f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:46:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.328+0000 7fcf43e00640 1 --2- 172.21.15.16:0/2247295970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf440abb00 0x7fcf44148f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:46:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.328+0000 7fcf43e00640 1 -- 172.21.15.16:0/2247295970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c009ae0 con 0x7fcf44156c50 2024-08-17T02:46:31.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.328+0000 7fcf43e00640 1 --2- 172.21.15.16:0/2247295970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf44156c50 0x7fcf44158700 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009f50 tx=0x7fcf3c036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:31.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.329+0000 7fcf3aa00640 1 -- 172.21.15.16:0/2247295970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c03c700 con 0x7fcf44156c50 2024-08-17T02:46:31.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.329+0000 7fcf3aa00640 1 -- 172.21.15.16:0/2247295970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c03c860 con 0x7fcf44156c50 2024-08-17T02:46:31.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.329+0000 7fcf3aa00640 1 -- 172.21.15.16:0/2247295970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c03cbe0 con 0x7fcf44156c50 2024-08-17T02:46:31.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.329+0000 7fcf4a600640 1 -- 172.21.15.16:0/2247295970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf44169630 con 0x7fcf44156c50 2024-08-17T02:46:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.329+0000 7fcf4a600640 1 -- 172.21.15.16:0/2247295970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf44169bf0 con 0x7fcf44156c50 2024-08-17T02:46:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.330+0000 7fcf4a600640 1 -- 172.21.15.16:0/2247295970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf44071050 con 0x7fcf44156c50 2024-08-17T02:46:31.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.330+0000 7fcf3aa00640 1 -- 172.21.15.16:0/2247295970 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fcf3c03b380 con 0x7fcf44156c50 2024-08-17T02:46:31.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.332+0000 7fcf3aa00640 1 --2- 172.21.15.16:0/2247295970 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcf24065150 0x7fcf24067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:46:31.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.332+0000 7fcf3aa00640 1 -- 172.21.15.16:0/2247295970 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fcf3c0a3670 con 0x7fcf44156c50 2024-08-17T02:46:31.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.332+0000 7fcf43400640 1 --2- 172.21.15.16:0/2247295970 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcf24065150 0x7fcf24067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:46:31.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.333+0000 7fcf43400640 1 --2- 172.21.15.16:0/2247295970 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcf24065150 0x7fcf24067600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcf440accc0 tx=0x7fcf34008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:46:31.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.336+0000 7fcf3aa00640 1 -- 172.21.15.16:0/2247295970 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcf3c071310 con 0x7fcf44156c50 2024-08-17T02:46:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:46:31.542+0000 7fcf4a600640 1 -- 172.21.15.16:0/2247295970 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000521811", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf440aef70 con 0x7fcf24065150 2024-08-17T02:46:33.725 INFO:tasks.ceph.mds.c.smithi016.stderr:2024-08-17T02:46:33.721+0000 7f73eb600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-17T02:46:49.957 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:46:49.962 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:47:20.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:47:20.076 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:47:50.176 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:47:50.182 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:48:20.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:48:20.286 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:48:31.238 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-08-17T02:48:31.239 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-08-17T02:48:31.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-17T02:48:31.336 DEBUG:teuthology.orchestra.run.smithi142:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-08-17T02:48:31.434 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-08-17T02:48:31.434 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-08-17T02:48:31.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:48:31.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-08-17T02:48:31.503 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:48:31.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:31.503 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-17T02:48:31.540 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-08-17T02:48:31.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-08-17T02:48:31.540 DEBUG:teuthology.orchestra.run.smithi142:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-08-17T02:48:31.605 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:48:31.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:31.605 DEBUG:teuthology.orchestra.run.smithi142:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-08-17T02:48:31.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:48:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.796+0000 7fb9c2800640 1 Processor -- start 2024-08-17T02:48:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.796+0000 7fb9c2800640 1 -- start start 2024-08-17T02:48:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.797+0000 7fb9c2800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9bc158d60 0x7fb9bc0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.797+0000 7fb9c2800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb9bc15cb30 0x7fb9bc0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.797+0000 7fb9c2800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9bc0b18d0 con 0x7fb9bc159130 2024-08-17T02:48:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.797+0000 7fb9c2800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9bc0b0d00 con 0x7fb9bc15cb30 2024-08-17T02:48:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.797+0000 7fb9c2800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc0b0fa0 con 0x7fb9bc158d60 2024-08-17T02:48:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.798+0000 7fb9baa00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb9bc15cb30 0x7fb9bc0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.798+0000 7fb9baa00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb9bc15cb30 0x7fb9bc0b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48206/0 (socket says 172.21.15.16:48206) 2024-08-17T02:48:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.798+0000 7fb9bbe00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb9bc159130 0x7fb9bc0c6d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44346/0 (socket says 172.21.15.16:44346) 2024-08-17T02:48:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.798+0000 7fb9baa00640 1 -- 172.21.15.16:0/2272285951 learned_addr learned my addr 172.21.15.16:0/2272285951 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.798+0000 7fb9bb400640 1 --2- 172.21.15.16:0/2272285951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9bc158d60 0x7fb9bc0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:31.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.798+0000 7fb9baa00640 1 -- 172.21.15.16:0/2272285951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9bc158d60 msgr2=0x7fb9bc0b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.798+0000 7fb9baa00640 1 --2- 172.21.15.16:0/2272285951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9bc158d60 0x7fb9bc0b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.798+0000 7fb9baa00640 1 -- 172.21.15.16:0/2272285951 >> v1:172.21.15.16:6789/0 conn(0x7fb9bc159130 legacy=0x7fb9bc0c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.798+0000 7fb9baa00640 1 -- 172.21.15.16:0/2272285951 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc0b1280 con 0x7fb9bc15cb30 2024-08-17T02:48:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.798+0000 7fb9ba000640 1 -- 172.21.15.16:0/2272285951 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761124787 0 0) 0x7fb9bc0b18d0 con 0x7fb9bc159130 2024-08-17T02:48:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.799+0000 7fb9baa00640 1 --2- 172.21.15.16:0/2272285951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc15cb30 0x7fb9bc0b22a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009910 tx=0x7fb9a80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85b8a27a22c3874 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.799+0000 7fb9ba000640 1 -- 172.21.15.16:0/2272285951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a802ae40 con 0x7fb9bc15cb30 2024-08-17T02:48:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.800+0000 7fb9ba000640 1 -- 172.21.15.16:0/2272285951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a8032440 con 0x7fb9bc15cb30 2024-08-17T02:48:31.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.800+0000 7fb9ba000640 1 -- 172.21.15.16:0/2272285951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a80327e0 con 0x7fb9bc15cb30 2024-08-17T02:48:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.800+0000 7fb9c2800640 1 -- 172.21.15.16:0/2272285951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc15cb30 msgr2=0x7fb9bc0b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.800+0000 7fb9c2800640 1 --2- 172.21.15.16:0/2272285951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc15cb30 0x7fb9bc0b22a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009910 tx=0x7fb9a80091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.800+0000 7fb9c2800640 1 -- 172.21.15.16:0/2272285951 shutdown_connections 2024-08-17T02:48:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.800+0000 7fb9c2800640 1 --2- 172.21.15.16:0/2272285951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc15cb30 0x7fb9bc0b22a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.800+0000 7fb9c2800640 1 --2- 172.21.15.16:0/2272285951 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9bc158d60 0x7fb9bc0b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.800+0000 7fb9c2800640 1 -- 172.21.15.16:0/2272285951 >> 172.21.15.16:0/2272285951 conn(0x7fb9bc0b3040 msgr2=0x7fb9bc0b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.800+0000 7fb9c2800640 1 -- 172.21.15.16:0/2272285951 shutdown_connections 2024-08-17T02:48:31.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.800+0000 7fb9c2800640 1 -- 172.21.15.16:0/2272285951 wait complete. 2024-08-17T02:48:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.801+0000 7fb9c2800640 1 Processor -- start 2024-08-17T02:48:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.802+0000 7fb9c2800640 1 -- start start 2024-08-17T02:48:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.802+0000 7fb9c2800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc158d60 0x7fb9bc142f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.802+0000 7fb9c2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9bc159130 0x7fb9bc143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.802+0000 7fb9c2800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9bc15cb30 0x7fb9bc099e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.802+0000 7fb9c2800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc0ab5f0 con 0x7fb9bc158d60 2024-08-17T02:48:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.802+0000 7fb9c2800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc143b00 con 0x7fb9bc159130 2024-08-17T02:48:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.802+0000 7fb9c2800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc0ab840 con 0x7fb9bc15cb30 2024-08-17T02:48:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.803+0000 7fb9bb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc158d60 0x7fb9bc142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.803+0000 7fb9bb400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc158d60 0x7fb9bc142f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48208/0 (socket says 172.21.15.16:48208) 2024-08-17T02:48:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.803+0000 7fb9bb400640 1 -- 172.21.15.16:0/3464732603 learned_addr learned my addr 172.21.15.16:0/3464732603 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:31.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.803+0000 7fb9baa00640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9bc159130 0x7fb9bc143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.803+0000 7fb9bbe00640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9bc15cb30 0x7fb9bc099e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:31.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.803+0000 7fb9bb400640 1 -- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9bc15cb30 msgr2=0x7fb9bc099e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.803+0000 7fb9bb400640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9bc15cb30 0x7fb9bc099e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.803+0000 7fb9bb400640 1 -- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9bc159130 msgr2=0x7fb9bc143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.803+0000 7fb9bb400640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9bc159130 0x7fb9bc143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.803+0000 7fb9bb400640 1 -- 172.21.15.16:0/3464732603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a8009450 con 0x7fb9bc158d60 2024-08-17T02:48:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.804+0000 7fb9bb400640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc158d60 0x7fb9bc142f20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00ea20 tx=0x7fb9ac0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.804+0000 7fb9b2a00640 1 -- 172.21.15.16:0/3464732603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac00ca10 con 0x7fb9bc158d60 2024-08-17T02:48:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.804+0000 7fb9b2a00640 1 -- 172.21.15.16:0/3464732603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ac00cb70 con 0x7fb9bc158d60 2024-08-17T02:48:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.804+0000 7fb9b2a00640 1 -- 172.21.15.16:0/3464732603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac013470 con 0x7fb9bc158d60 2024-08-17T02:48:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.804+0000 7fb9c2800640 1 -- 172.21.15.16:0/3464732603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9bc0abb20 con 0x7fb9bc158d60 2024-08-17T02:48:31.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.804+0000 7fb9c2800640 1 -- 172.21.15.16:0/3464732603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9bc0ac0e0 con 0x7fb9bc158d60 2024-08-17T02:48:31.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.805+0000 7fb9c2800640 1 -- 172.21.15.16:0/3464732603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9bc064db0 con 0x7fb9bc158d60 2024-08-17T02:48:31.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.806+0000 7fb9b2a00640 1 -- 172.21.15.16:0/3464732603 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fb9ac013860 con 0x7fb9bc158d60 2024-08-17T02:48:31.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.807+0000 7fb9b2a00640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb99c065150 0x7fb99c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:31.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.807+0000 7fb9b2a00640 1 -- 172.21.15.16:0/3464732603 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fb9ac082b20 con 0x7fb9bc158d60 2024-08-17T02:48:31.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.807+0000 7fb9baa00640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb99c065150 0x7fb99c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:31.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.808+0000 7fb9baa00640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb99c065150 0x7fb99c067600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8034000 tx=0x7fb9a8032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:31.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:31.811+0000 7fb9b2a00640 1 -- 172.21.15.16:0/3464732603 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fb9ac050800 con 0x7fb9bc158d60 2024-08-17T02:48:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.032+0000 7fb9c2800640 1 -- 172.21.15.16:0/3464732603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9bc069af0 con 0x7fb9bc158d60 2024-08-17T02:48:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.032+0000 7fb9b2a00640 1 -- 172.21.15.16:0/3464732603 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5687 (secure 0 0 0) 0x7fb9ac053e20 con 0x7fb9bc158d60 2024-08-17T02:48:32.032 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:32.033 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2889208700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2889208700},{"type":"v1","addr":"172.21.15.142:6837","nonce":2889208700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4287834697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4287834697},{"type":"v1","addr":"172.21.15.142:6835","nonce":4287834697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:16.190199+0000","modified":"2024-08-17T02:46:19.780153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6835/219684854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":219684854},{"type":"v1","addr":"172.21.15.16:6835","nonce":219684854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:28.938945+0000","modified":"2024-08-17T02:46:30.943869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6834/3115083427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3115083427},{"type":"v1","addr":"172.21.15.16:6834","nonce":3115083427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000521811","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:48:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2024-08-17T02:48:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.034+0000 7fb9c2800640 1 -- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb99c065150 msgr2=0x7fb99c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.034+0000 7fb9c2800640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb99c065150 0x7fb99c067600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8034000 tx=0x7fb9a8032000 comp rx=0 tx=0).stop 2024-08-17T02:48:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.034+0000 7fb9c2800640 1 -- 172.21.15.16:0/3464732603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc158d60 msgr2=0x7fb9bc142f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.034+0000 7fb9c2800640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc158d60 0x7fb9bc142f20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00ea20 tx=0x7fb9ac0090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.034+0000 7fb9c2800640 1 -- 172.21.15.16:0/3464732603 shutdown_connections 2024-08-17T02:48:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.034+0000 7fb9c2800640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fb99c065150 0x7fb99c067600 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.034+0000 7fb9c2800640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fb9bc15cb30 0x7fb9bc099e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.034+0000 7fb9c2800640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9bc159130 0x7fb9bc143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.034+0000 7fb9c2800640 1 --2- 172.21.15.16:0/3464732603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc158d60 0x7fb9bc142f20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.034+0000 7fb9c2800640 1 -- 172.21.15.16:0/3464732603 >> 172.21.15.16:0/3464732603 conn(0x7fb9bc0b3040 msgr2=0x7fb9bc1595b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.034+0000 7fb9c2800640 1 -- 172.21.15.16:0/3464732603 shutdown_connections 2024-08-17T02:48:32.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.035+0000 7fb9c2800640 1 -- 172.21.15.16:0/3464732603 wait complete. 2024-08-17T02:48:32.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:48:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.201+0000 7f44f3000640 1 Processor -- start 2024-08-17T02:48:32.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.201+0000 7f44f3000640 1 -- start start 2024-08-17T02:48:32.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.202+0000 7f44f3000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f44ec156ad0 0x7f44ec0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.202+0000 7f44f3000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f44ec156ea0 0x7f44ec06d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.202+0000 7f44f3000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ec0caaf0 con 0x7f44ec06dab0 2024-08-17T02:48:32.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.202+0000 7f44f3000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec0ca900 con 0x7f44ec156ad0 2024-08-17T02:48:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.202+0000 7f44f3000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec0cc560 con 0x7f44ec156ea0 2024-08-17T02:48:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.202+0000 7f44f1c00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f44ec156ad0 0x7f44ec0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.202+0000 7f44f1c00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f44ec156ad0 0x7f44ec0cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48212/0 (socket says 172.21.15.16:48212) 2024-08-17T02:48:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.202+0000 7f44f1c00640 1 -- 172.21.15.16:0/1800412771 learned_addr learned my addr 172.21.15.16:0/1800412771 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.202+0000 7f44f1200640 1 --2- 172.21.15.16:0/1800412771 >> v2:172.21.15.142:3300/0 conn(0x7f44ec156ea0 0x7f44ec06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.203+0000 7f44f1c00640 1 -- 172.21.15.16:0/1800412771 >> v2:172.21.15.142:3300/0 conn(0x7f44ec156ea0 msgr2=0x7f44ec06d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.203+0000 7f44f1c00640 1 --2- 172.21.15.16:0/1800412771 >> v2:172.21.15.142:3300/0 conn(0x7f44ec156ea0 0x7f44ec06d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.203+0000 7f44f1c00640 1 -- 172.21.15.16:0/1800412771 >> v1:172.21.15.142:6789/0 conn(0x7f44ec06dab0 legacy=0x7f44ec0c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.203+0000 7f44f1c00640 1 -- 172.21.15.16:0/1800412771 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec157300 con 0x7f44ec156ad0 2024-08-17T02:48:32.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.203+0000 7f44f1200640 1 --2- 172.21.15.16:0/1800412771 >> v2:172.21.15.142:3300/0 conn(0x7f44ec156ea0 0x7f44ec06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.204+0000 7f44f1c00640 1 --2- 172.21.15.16:0/1800412771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44ec156ad0 0x7f44ec0cbea0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f44ec06f430 tx=0x7f44e00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61983377185671c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.204+0000 7f44ebe00640 1 -- 172.21.15.16:0/1800412771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e002ae50 con 0x7f44ec156ad0 2024-08-17T02:48:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.204+0000 7f44ebe00640 1 -- 172.21.15.16:0/1800412771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e0032440 con 0x7f44ec156ad0 2024-08-17T02:48:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.204+0000 7f44ebe00640 1 -- 172.21.15.16:0/1800412771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0032820 con 0x7f44ec156ad0 2024-08-17T02:48:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.205+0000 7f44f3000640 1 -- 172.21.15.16:0/1800412771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44ec156ad0 msgr2=0x7f44ec0cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.205+0000 7f44f3000640 1 --2- 172.21.15.16:0/1800412771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44ec156ad0 0x7f44ec0cbea0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f44ec06f430 tx=0x7f44e00091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.205+0000 7f44f3000640 1 -- 172.21.15.16:0/1800412771 shutdown_connections 2024-08-17T02:48:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.205+0000 7f44f3000640 1 --2- 172.21.15.16:0/1800412771 >> v2:172.21.15.142:3300/0 conn(0x7f44ec156ea0 0x7f44ec06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.205+0000 7f44f3000640 1 --2- 172.21.15.16:0/1800412771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44ec156ad0 0x7f44ec0cbea0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.205+0000 7f44f3000640 1 -- 172.21.15.16:0/1800412771 >> 172.21.15.16:0/1800412771 conn(0x7f44ec067750 msgr2=0x7f44ec0cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:32.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.205+0000 7f44f3000640 1 -- 172.21.15.16:0/1800412771 shutdown_connections 2024-08-17T02:48:32.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.205+0000 7f44f3000640 1 -- 172.21.15.16:0/1800412771 wait complete. 2024-08-17T02:48:32.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.206+0000 7f44f3000640 1 Processor -- start 2024-08-17T02:48:32.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.206+0000 7f44f3000640 1 -- start start 2024-08-17T02:48:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.207+0000 7f44f3000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44ec06dab0 0x7f44ec0ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.207+0000 7f44f3000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44ec156ad0 0x7f44ec16a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.207+0000 7f44f3000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44ec156ea0 0x7f44ec0c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.207+0000 7f44f3000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0af830 con 0x7f44ec06dab0 2024-08-17T02:48:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.207+0000 7f44f3000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0aecf0 con 0x7f44ec156ad0 2024-08-17T02:48:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.207+0000 7f44f3000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0aef90 con 0x7f44ec156ea0 2024-08-17T02:48:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.208+0000 7f44f1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44ec06dab0 0x7f44ec0ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.208+0000 7f44f1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44ec06dab0 0x7f44ec0ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48224/0 (socket says 172.21.15.16:48224) 2024-08-17T02:48:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.208+0000 7f44f1c00640 1 -- 172.21.15.16:0/820071699 learned_addr learned my addr 172.21.15.16:0/820071699 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:32.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.208+0000 7f44f1200640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44ec156ad0 0x7f44ec16a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.208+0000 7f44f2600640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44ec156ea0 0x7f44ec0c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.208+0000 7f44f1c00640 1 -- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44ec156ea0 msgr2=0x7f44ec0c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.208+0000 7f44f1c00640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44ec156ea0 0x7f44ec0c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.208+0000 7f44f1c00640 1 -- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44ec156ad0 msgr2=0x7f44ec16a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.208+0000 7f44f1c00640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44ec156ad0 0x7f44ec16a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.208+0000 7f44f1c00640 1 -- 172.21.15.16:0/820071699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e0009450 con 0x7f44ec06dab0 2024-08-17T02:48:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.208+0000 7f44f1c00640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44ec06dab0 0x7f44ec0ae7b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f44e0035040 tx=0x7f44e0032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.208+0000 7f44e8c00640 1 -- 172.21.15.16:0/820071699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0031a60 con 0x7f44ec06dab0 2024-08-17T02:48:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.209+0000 7f44e8c00640 1 -- 172.21.15.16:0/820071699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e0031c00 con 0x7f44ec06dab0 2024-08-17T02:48:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.209+0000 7f44f3000640 1 -- 172.21.15.16:0/820071699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec0af270 con 0x7f44ec06dab0 2024-08-17T02:48:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.209+0000 7f44e8c00640 1 -- 172.21.15.16:0/820071699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e003d440 con 0x7f44ec06dab0 2024-08-17T02:48:32.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.209+0000 7f44f3000640 1 -- 172.21.15.16:0/820071699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec1597e0 con 0x7f44ec06dab0 2024-08-17T02:48:32.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.210+0000 7f44f3000640 1 -- 172.21.15.16:0/820071699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ec151f90 con 0x7f44ec06dab0 2024-08-17T02:48:32.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.211+0000 7f44e8c00640 1 -- 172.21.15.16:0/820071699 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f44e0039040 con 0x7f44ec06dab0 2024-08-17T02:48:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.212+0000 7f44e8c00640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f44cc065150 0x7f44cc067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.212+0000 7f44e8c00640 1 -- 172.21.15.16:0/820071699 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f44e00a2810 con 0x7f44ec06dab0 2024-08-17T02:48:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.212+0000 7f44f1200640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f44cc065150 0x7f44cc067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.213+0000 7f44f1200640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f44cc065150 0x7f44cc067600 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0c8400 tx=0x7f44d8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:32.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.216+0000 7f44e8c00640 1 -- 172.21.15.16:0/820071699 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f44e00705b0 con 0x7f44ec06dab0 2024-08-17T02:48:32.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.398+0000 7f44f3000640 1 -- 172.21.15.16:0/820071699 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44ec071050 con 0x7f44ec06dab0 2024-08-17T02:48:32.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.399+0000 7f44e8c00640 1 -- 172.21.15.16:0/820071699 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18788 (secure 0 0 0) 0x7f44e0073bd0 con 0x7f44ec06dab0 2024-08-17T02:48:32.400 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:32.400 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:47:07.371280+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:46:14.322032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:46:15.170652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000521811.meta","create_time":"2024-08-17T02:46:27.102528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000521811"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000521811.data","create_time":"2024-08-17T02:46:27.975177+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000521811"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.e","mappings":[{"from":4,"to":5}]},{"pgid":"5.27","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]},{"pgid":"5.37","mappings":[{"from":4,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:48:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.401+0000 7f44f3000640 1 -- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f44cc065150 msgr2=0x7f44cc067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.401+0000 7f44f3000640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f44cc065150 0x7f44cc067600 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0c8400 tx=0x7f44d8008040 comp rx=0 tx=0).stop 2024-08-17T02:48:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.401+0000 7f44f3000640 1 -- 172.21.15.16:0/820071699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44ec06dab0 msgr2=0x7f44ec0ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.401+0000 7f44f3000640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44ec06dab0 0x7f44ec0ae7b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f44e0035040 tx=0x7f44e0032000 comp rx=0 tx=0).stop 2024-08-17T02:48:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.401+0000 7f44f3000640 1 -- 172.21.15.16:0/820071699 shutdown_connections 2024-08-17T02:48:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.401+0000 7f44f3000640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f44cc065150 0x7f44cc067600 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.401+0000 7f44f3000640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f44ec156ea0 0x7f44ec0c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.401+0000 7f44f3000640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44ec156ad0 0x7f44ec16a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.401+0000 7f44f3000640 1 --2- 172.21.15.16:0/820071699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44ec06dab0 0x7f44ec0ae7b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.401+0000 7f44f3000640 1 -- 172.21.15.16:0/820071699 >> 172.21.15.16:0/820071699 conn(0x7f44ec067750 msgr2=0x7f44ec069180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:32.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.401+0000 7f44f3000640 1 -- 172.21.15.16:0/820071699 shutdown_connections 2024-08-17T02:48:32.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.401+0000 7f44f3000640 1 -- 172.21.15.16:0/820071699 wait complete. 2024-08-17T02:48:32.418 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:45:14.291903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-17T02:46:14.322032+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-08-17T02:46:15.170652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs.volume_0000000000521811.meta', 'create_time': '2024-08-17T02:46:27.102528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000521811'}}}, {'pool': 7, 'pool_name': 'cephfs.volume_0000000000521811.data', 'create_time': '2024-08-17T02:46:27.975177+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000521811'}}}] 2024-08-17T02:48:32.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:48:32.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.573+0000 7f0cd9600640 1 Processor -- start 2024-08-17T02:48:32.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.573+0000 7f0cd9600640 1 -- start start 2024-08-17T02:48:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd9600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0cd4154770 0x7f0cd40cb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd9600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0cd406e3e0 0x7f0cd4157970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd9600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd4157eb0 con 0x7f0cd41543a0 2024-08-17T02:48:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd9600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd40ca160 con 0x7f0cd4154770 2024-08-17T02:48:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd9600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd40ca440 con 0x7f0cd406e3e0 2024-08-17T02:48:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd3400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0cd4154770 0x7f0cd40cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd3400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0cd4154770 0x7f0cd40cb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48240/0 (socket says 172.21.15.16:48240) 2024-08-17T02:48:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd3400640 1 -- 172.21.15.16:0/2759386821 learned_addr learned my addr 172.21.15.16:0/2759386821 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd8c00640 1 --2- 172.21.15.16:0/2759386821 >> v2:172.21.15.142:3300/0 conn(0x7f0cd406e3e0 0x7f0cd4157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd3400640 1 -- 172.21.15.16:0/2759386821 >> v2:172.21.15.142:3300/0 conn(0x7f0cd406e3e0 msgr2=0x7f0cd4157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd3400640 1 --2- 172.21.15.16:0/2759386821 >> v2:172.21.15.142:3300/0 conn(0x7f0cd406e3e0 0x7f0cd4157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd3400640 1 -- 172.21.15.16:0/2759386821 >> v1:172.21.15.142:6789/0 conn(0x7f0cd41543a0 legacy=0x7f0cd406dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd3400640 1 -- 172.21.15.16:0/2759386821 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40ca720 con 0x7f0cd4154770 2024-08-17T02:48:32.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.575+0000 7f0cd8c00640 1 --2- 172.21.15.16:0/2759386821 >> v2:172.21.15.142:3300/0 conn(0x7f0cd406e3e0 0x7f0cd4157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.576+0000 7f0cd3400640 1 --2- 172.21.15.16:0/2759386821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd4154770 0x7f0cd40cb200 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40b1b60 tx=0x7f0cc00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81f8f369f6bb76e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.577+0000 7f0cd2a00640 1 -- 172.21.15.16:0/2759386821 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc002ae80 con 0x7f0cd4154770 2024-08-17T02:48:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.577+0000 7f0cd2a00640 1 -- 172.21.15.16:0/2759386821 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc0032440 con 0x7f0cd4154770 2024-08-17T02:48:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.577+0000 7f0cd2a00640 1 -- 172.21.15.16:0/2759386821 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0032800 con 0x7f0cd4154770 2024-08-17T02:48:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.577+0000 7f0cd9600640 1 -- 172.21.15.16:0/2759386821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd4154770 msgr2=0x7f0cd40cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.577+0000 7f0cd9600640 1 --2- 172.21.15.16:0/2759386821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd4154770 0x7f0cd40cb200 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40b1b60 tx=0x7f0cc00091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.577+0000 7f0cd9600640 1 -- 172.21.15.16:0/2759386821 shutdown_connections 2024-08-17T02:48:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.577+0000 7f0cd9600640 1 --2- 172.21.15.16:0/2759386821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd4154770 0x7f0cd40cb200 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.577+0000 7f0cd9600640 1 --2- 172.21.15.16:0/2759386821 >> v2:172.21.15.142:3300/0 conn(0x7f0cd406e3e0 0x7f0cd4157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.577+0000 7f0cd9600640 1 -- 172.21.15.16:0/2759386821 >> 172.21.15.16:0/2759386821 conn(0x7f0cd4067750 msgr2=0x7f0cd406c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.577+0000 7f0cd9600640 1 -- 172.21.15.16:0/2759386821 shutdown_connections 2024-08-17T02:48:32.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.577+0000 7f0cd9600640 1 -- 172.21.15.16:0/2759386821 wait complete. 2024-08-17T02:48:32.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.579+0000 7f0cd9600640 1 Processor -- start 2024-08-17T02:48:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.579+0000 7f0cd9600640 1 -- start start 2024-08-17T02:48:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd9600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd406e3e0 0x7f0cd4137820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd9600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cd41543a0 0x7f0cd415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd9600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0cd4154770 0x7f0cd415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd9600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd414aea0 con 0x7f0cd406e3e0 2024-08-17T02:48:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd9600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd4137d60 con 0x7f0cd41543a0 2024-08-17T02:48:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd9600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd414b700 con 0x7f0cd4154770 2024-08-17T02:48:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd406e3e0 0x7f0cd4137820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd406e3e0 0x7f0cd4137820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48246/0 (socket says 172.21.15.16:48246) 2024-08-17T02:48:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd3e00640 1 -- 172.21.15.16:0/3550211863 learned_addr learned my addr 172.21.15.16:0/3550211863 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:32.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd3400640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cd41543a0 0x7f0cd415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd8c00640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0cd4154770 0x7f0cd415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd3e00640 1 -- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0cd4154770 msgr2=0x7f0cd415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd3e00640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0cd4154770 0x7f0cd415a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd3e00640 1 -- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cd41543a0 msgr2=0x7f0cd415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd3e00640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cd41543a0 0x7f0cd415a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.580+0000 7f0cd3e00640 1 -- 172.21.15.16:0/3550211863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc0009450 con 0x7f0cd406e3e0 2024-08-17T02:48:32.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.581+0000 7f0cd3e00640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd406e3e0 0x7f0cd4137820 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400c4c0 tx=0x7f0cc4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.581+0000 7f0ccb400640 1 -- 172.21.15.16:0/3550211863 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4011590 con 0x7f0cd406e3e0 2024-08-17T02:48:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.581+0000 7f0ccb400640 1 -- 172.21.15.16:0/3550211863 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc4011710 con 0x7f0cd406e3e0 2024-08-17T02:48:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.581+0000 7f0ccb400640 1 -- 172.21.15.16:0/3550211863 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4011ad0 con 0x7f0cd406e3e0 2024-08-17T02:48:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.581+0000 7f0cd9600640 1 -- 172.21.15.16:0/3550211863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd406a260 con 0x7f0cd406e3e0 2024-08-17T02:48:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.581+0000 7f0cd9600640 1 -- 172.21.15.16:0/3550211863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd414b390 con 0x7f0cd406e3e0 2024-08-17T02:48:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.582+0000 7f0cd9600640 1 -- 172.21.15.16:0/3550211863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd4071050 con 0x7f0cd406e3e0 2024-08-17T02:48:32.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.583+0000 7f0ccb400640 1 -- 172.21.15.16:0/3550211863 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f0cc401b3b0 con 0x7f0cd406e3e0 2024-08-17T02:48:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.584+0000 7f0ccb400640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0cb4065180 0x7f0cb4067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.584+0000 7f0ccb400640 1 -- 172.21.15.16:0/3550211863 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f0cc4081c40 con 0x7f0cd406e3e0 2024-08-17T02:48:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.584+0000 7f0cd3400640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0cb4065180 0x7f0cb4067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.585+0000 7f0cd3400640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0cb4065180 0x7f0cb4067630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0034000 tx=0x7f0cc0032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:32.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.588+0000 7f0ccb400640 1 -- 172.21.15.16:0/3550211863 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0cc404f9e0 con 0x7f0cd406e3e0 2024-08-17T02:48:32.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.767+0000 7f0cd9600640 1 -- 172.21.15.16:0/3550211863 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0cd40b0f30 con 0x7f0cd406e3e0 2024-08-17T02:48:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.768+0000 7f0ccb400640 1 -- 172.21.15.16:0/3550211863 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0cc4017070 con 0x7f0cd406e3e0 2024-08-17T02:48:32.768 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-17T02:48:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.769+0000 7f0cd9600640 1 -- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0cb4065180 msgr2=0x7f0cb4067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.769+0000 7f0cd9600640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0cb4065180 0x7f0cb4067630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0034000 tx=0x7f0cc0032000 comp rx=0 tx=0).stop 2024-08-17T02:48:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.769+0000 7f0cd9600640 1 -- 172.21.15.16:0/3550211863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd406e3e0 msgr2=0x7f0cd4137820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.769+0000 7f0cd9600640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd406e3e0 0x7f0cd4137820 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400c4c0 tx=0x7f0cc4007ef0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.770+0000 7f0cd9600640 1 -- 172.21.15.16:0/3550211863 shutdown_connections 2024-08-17T02:48:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.770+0000 7f0cd9600640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0cb4065180 0x7f0cb4067630 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.770+0000 7f0cd9600640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0cd4154770 0x7f0cd415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.770+0000 7f0cd9600640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0cd41543a0 0x7f0cd415a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.770+0000 7f0cd9600640 1 --2- 172.21.15.16:0/3550211863 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0cd406e3e0 0x7f0cd4137820 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.770+0000 7f0cd9600640 1 -- 172.21.15.16:0/3550211863 >> 172.21.15.16:0/3550211863 conn(0x7f0cd4067750 msgr2=0x7f0cd4154b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.770+0000 7f0cd9600640 1 -- 172.21.15.16:0/3550211863 shutdown_connections 2024-08-17T02:48:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.770+0000 7f0cd9600640 1 -- 172.21.15.16:0/3550211863 wait complete. 2024-08-17T02:48:32.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-17T02:48:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.940+0000 7fcc0fe00640 1 Processor -- start 2024-08-17T02:48:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.940+0000 7fcc0fe00640 1 -- start start 2024-08-17T02:48:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.942+0000 7fcc0fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc100673e0 0x7fcc10163b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.942+0000 7fcc0fe00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1006e660 con 0x7fcc10158940 2024-08-17T02:48:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.942+0000 7fcc0fe00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc100a9b70 con 0x7fcc10158d10 2024-08-17T02:48:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.942+0000 7fcc0fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1006da50 con 0x7fcc100673e0 2024-08-17T02:48:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.942+0000 7fcc0ea00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcc10158940 0x7fcc1015d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44362/0 (socket says 172.21.15.16:44362) 2024-08-17T02:48:32.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.942+0000 7fcc0ea00640 1 -- 172.21.15.16:0/1076081602 learned_addr learned my addr 172.21.15.16:0/1076081602 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.942+0000 7fcc0f400640 1 --2- 172.21.15.16:0/1076081602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc100673e0 0x7fcc10163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.942+0000 7fcc0e000640 1 -- 172.21.15.16:0/1076081602 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825753880 0 0) 0x7fcc1006e660 con 0x7fcc10158940 2024-08-17T02:48:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.943+0000 7fcc0e000640 1 -- 172.21.15.16:0/1076081602 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbf40034e0 con 0x7fcc10158940 2024-08-17T02:48:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.943+0000 7fcc0e000640 1 -- 172.21.15.16:0/1076081602 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070889251 0 0) 0x7fcc100a9b70 con 0x7fcc10158d10 2024-08-17T02:48:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.943+0000 7fcc0e000640 1 -- 172.21.15.16:0/1076081602 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc1006e660 con 0x7fcc10158d10 2024-08-17T02:48:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.943+0000 7fcc0f400640 1 -- 172.21.15.16:0/1076081602 >> v1:172.21.15.142:6789/0 conn(0x7fcc10158d10 legacy=0x7fcc10066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.943+0000 7fcc0f400640 1 -- 172.21.15.16:0/1076081602 >> v1:172.21.15.16:6789/0 conn(0x7fcc10158940 legacy=0x7fcc1015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.943+0000 7fcc0f400640 1 -- 172.21.15.16:0/1076081602 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1006dd30 con 0x7fcc100673e0 2024-08-17T02:48:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.944+0000 7fcc0f400640 1 --2- 172.21.15.16:0/1076081602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc100673e0 0x7fcc10163b00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009e90 tx=0x7fcc08009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff0e248ff6d01962 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.944+0000 7fcc0e000640 1 -- 172.21.15.16:0/1076081602 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0802aea0 con 0x7fcc100673e0 2024-08-17T02:48:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.944+0000 7fcc0e000640 1 -- 172.21.15.16:0/1076081602 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0802d050 con 0x7fcc100673e0 2024-08-17T02:48:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.944+0000 7fcc0e000640 1 -- 172.21.15.16:0/1076081602 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08007780 con 0x7fcc100673e0 2024-08-17T02:48:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.945+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1076081602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc100673e0 msgr2=0x7fcc10163b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.945+0000 7fcc0fe00640 1 --2- 172.21.15.16:0/1076081602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc100673e0 0x7fcc10163b00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009e90 tx=0x7fcc08009780 comp rx=0 tx=0).stop 2024-08-17T02:48:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.945+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1076081602 shutdown_connections 2024-08-17T02:48:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.945+0000 7fcc0fe00640 1 --2- 172.21.15.16:0/1076081602 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc100673e0 0x7fcc10163b00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.945+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1076081602 >> 172.21.15.16:0/1076081602 conn(0x7fcc10067de0 msgr2=0x7fcc100681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.945+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1076081602 shutdown_connections 2024-08-17T02:48:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.945+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1076081602 wait complete. 2024-08-17T02:48:32.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.946+0000 7fcc0fe00640 1 Processor -- start 2024-08-17T02:48:32.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.946+0000 7fcc0fe00640 1 -- start start 2024-08-17T02:48:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.947+0000 7fcc0fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc100673e0 0x7fcc101454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.947+0000 7fcc0fe00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc10158940 0x7fcc10169bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.947+0000 7fcc0fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc10158d10 0x7fcc10069790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.947+0000 7fcc0fe00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc10145a20 con 0x7fcc10158d10 2024-08-17T02:48:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.947+0000 7fcc0fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1015f0c0 con 0x7fcc100673e0 2024-08-17T02:48:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.947+0000 7fcc0fe00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1016a100 con 0x7fcc10158940 2024-08-17T02:48:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.947+0000 7fcc0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc10158d10 0x7fcc10069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.947+0000 7fcc0f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc10158d10 0x7fcc10069790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48252/0 (socket says 172.21.15.16:48252) 2024-08-17T02:48:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.947+0000 7fcc0f400640 1 -- 172.21.15.16:0/1641726186 learned_addr learned my addr 172.21.15.16:0/1641726186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.948+0000 7fcc0ea00640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc100673e0 0x7fcc101454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.948+0000 7fcc07e00640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc10158940 0x7fcc10169bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.948+0000 7fcc0f400640 1 -- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc10158940 msgr2=0x7fcc10169bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.948+0000 7fcc0f400640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc10158940 0x7fcc10169bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.948+0000 7fcc0f400640 1 -- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc100673e0 msgr2=0x7fcc101454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.948+0000 7fcc0f400640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc100673e0 0x7fcc101454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.948+0000 7fcc0f400640 1 -- 172.21.15.16:0/1641726186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc08009a20 con 0x7fcc10158d10 2024-08-17T02:48:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.948+0000 7fcc07e00640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc10158940 0x7fcc10169bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:32.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.948+0000 7fcc0f400640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc10158d10 0x7fcc10069790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcc080099f0 tx=0x7fcc08037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:32.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.949+0000 7fcc06000640 1 -- 172.21.15.16:0/1641726186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0802aea0 con 0x7fcc10158d10 2024-08-17T02:48:32.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.949+0000 7fcc06000640 1 -- 172.21.15.16:0/1641726186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc08031ac0 con 0x7fcc10158d10 2024-08-17T02:48:32.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.949+0000 7fcc06000640 1 -- 172.21.15.16:0/1641726186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08031e60 con 0x7fcc10158d10 2024-08-17T02:48:32.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.949+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1641726186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1015e4b0 con 0x7fcc10158d10 2024-08-17T02:48:32.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.949+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1641726186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1015ea70 con 0x7fcc10158d10 2024-08-17T02:48:32.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.950+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1641726186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc10071050 con 0x7fcc10158d10 2024-08-17T02:48:32.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.950+0000 7fcc06000640 1 -- 172.21.15.16:0/1641726186 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fcc08003020 con 0x7fcc10158d10 2024-08-17T02:48:32.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.951+0000 7fcc06000640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcbec065180 0x7fcbec067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:32.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.951+0000 7fcc06000640 1 -- 172.21.15.16:0/1641726186 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fcc080a3ea0 con 0x7fcc10158d10 2024-08-17T02:48:32.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.952+0000 7fcc0ea00640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcbec065180 0x7fcbec067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:32.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.952+0000 7fcc0ea00640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcbec065180 0x7fcbec067630 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcc00005840 tx=0x7fcc00009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:32.955+0000 7fcc06000640 1 -- 172.21.15.16:0/1641726186 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcc08071b80 con 0x7fcc10158d10 2024-08-17T02:48:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.137+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1641726186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc1015f200 con 0x7fcc10158d10 2024-08-17T02:48:33.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.137+0000 7fcc06000640 1 -- 172.21.15.16:0/1641726186 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc080751a0 con 0x7fcc10158d10 2024-08-17T02:48:33.138 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:48:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.139+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcbec065180 msgr2=0x7fcbec067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.139+0000 7fcc0fe00640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcbec065180 0x7fcbec067630 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcc00005840 tx=0x7fcc00009040 comp rx=0 tx=0).stop 2024-08-17T02:48:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.139+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1641726186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc10158d10 msgr2=0x7fcc10069790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.139+0000 7fcc0fe00640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc10158d10 0x7fcc10069790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcc080099f0 tx=0x7fcc08037000 comp rx=0 tx=0).stop 2024-08-17T02:48:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.139+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1641726186 shutdown_connections 2024-08-17T02:48:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.139+0000 7fcc0fe00640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcbec065180 0x7fcbec067630 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.139+0000 7fcc0fe00640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc10158d10 0x7fcc10069790 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.139+0000 7fcc0fe00640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcc10158940 0x7fcc10169bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.139+0000 7fcc0fe00640 1 --2- 172.21.15.16:0/1641726186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc100673e0 0x7fcc101454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.139+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1641726186 >> 172.21.15.16:0/1641726186 conn(0x7fcc10067de0 msgr2=0x7fcc100af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.139+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1641726186 shutdown_connections 2024-08-17T02:48:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.139+0000 7fcc0fe00640 1 -- 172.21.15.16:0/1641726186 wait complete. 2024-08-17T02:48:33.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-17T02:48:33.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.309+0000 7f152ee00640 1 Processor -- start 2024-08-17T02:48:33.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.309+0000 7f152ee00640 1 -- start start 2024-08-17T02:48:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.310+0000 7f152ee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15281590b0 0x7f15280c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.310+0000 7f152ee00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152806d790 con 0x7f15280c9110 2024-08-17T02:48:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.310+0000 7f152ee00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152806cb80 con 0x7f1528158ce0 2024-08-17T02:48:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.310+0000 7f152ee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15280c95b0 con 0x7f15281590b0 2024-08-17T02:48:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.311+0000 7f152ce00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f15280c9110 0x7f15280aee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44372/0 (socket says 172.21.15.16:44372) 2024-08-17T02:48:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.311+0000 7f152ce00640 1 -- 172.21.15.16:0/3147055310 learned_addr learned my addr 172.21.15.16:0/3147055310 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:33.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.311+0000 7f1527400640 1 --2- 172.21.15.16:0/3147055310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15281590b0 0x7f15280c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.311+0000 7f1526a00640 1 -- 172.21.15.16:0/3147055310 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336904124 0 0) 0x7f152806d790 con 0x7f15280c9110 2024-08-17T02:48:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.311+0000 7f1526a00640 1 -- 172.21.15.16:0/3147055310 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f150c0034e0 con 0x7f15280c9110 2024-08-17T02:48:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.311+0000 7f1526a00640 1 -- 172.21.15.16:0/3147055310 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559669513 0 0) 0x7f152806cb80 con 0x7f1528158ce0 2024-08-17T02:48:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.311+0000 7f1527400640 1 -- 172.21.15.16:0/3147055310 >> v1:172.21.15.142:6789/0 conn(0x7f1528158ce0 legacy=0x7f152806a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.311+0000 7f1527400640 1 -- 172.21.15.16:0/3147055310 >> v1:172.21.15.16:6789/0 conn(0x7f15280c9110 legacy=0x7f15280aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.311+0000 7f1527400640 1 -- 172.21.15.16:0/3147055310 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152806ce60 con 0x7f15281590b0 2024-08-17T02:48:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.312+0000 7f1527400640 1 --2- 172.21.15.16:0/3147055310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15281590b0 0x7f15280c8bd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1514009880 tx=0x7f15140090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c125b492a0508075 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:33.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.313+0000 7f1526a00640 1 -- 172.21.15.16:0/3147055310 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151402ae10 con 0x7f15281590b0 2024-08-17T02:48:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.313+0000 7f1526a00640 1 -- 172.21.15.16:0/3147055310 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1514032440 con 0x7f15281590b0 2024-08-17T02:48:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.313+0000 7f1526a00640 1 -- 172.21.15.16:0/3147055310 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140327e0 con 0x7f15281590b0 2024-08-17T02:48:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.313+0000 7f152ee00640 1 -- 172.21.15.16:0/3147055310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15281590b0 msgr2=0x7f15280c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.313+0000 7f152ee00640 1 --2- 172.21.15.16:0/3147055310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15281590b0 0x7f15280c8bd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1514009880 tx=0x7f15140090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.313+0000 7f152ee00640 1 -- 172.21.15.16:0/3147055310 shutdown_connections 2024-08-17T02:48:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.313+0000 7f152ee00640 1 --2- 172.21.15.16:0/3147055310 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f15281590b0 0x7f15280c8bd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.313+0000 7f152ee00640 1 -- 172.21.15.16:0/3147055310 >> 172.21.15.16:0/3147055310 conn(0x7f1528067750 msgr2=0x7f15280c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.313+0000 7f152ee00640 1 -- 172.21.15.16:0/3147055310 shutdown_connections 2024-08-17T02:48:33.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.313+0000 7f152ee00640 1 -- 172.21.15.16:0/3147055310 wait complete. 2024-08-17T02:48:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.315+0000 7f152ee00640 1 Processor -- start 2024-08-17T02:48:33.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.315+0000 7f152ee00640 1 -- start start 2024-08-17T02:48:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15280c9110 0x7f15280ab290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1528158ce0 0x7f1528157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15281590b0 0x7f15281543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:33.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15280ab990 con 0x7f15281590b0 2024-08-17T02:48:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15280abc70 con 0x7f15280c9110 2024-08-17T02:48:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15280abf50 con 0x7f1528158ce0 2024-08-17T02:48:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15281590b0 0x7f15281543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15281590b0 0x7f15281543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48258/0 (socket says 172.21.15.16:48258) 2024-08-17T02:48:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ce00640 1 -- 172.21.15.16:0/4223949002 learned_addr learned my addr 172.21.15.16:0/4223949002 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f1527400640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1528158ce0 0x7f1528157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f1527e00640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15280c9110 0x7f15280ab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ce00640 1 -- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1528158ce0 msgr2=0x7f1528157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ce00640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1528158ce0 0x7f1528157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ce00640 1 -- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15280c9110 msgr2=0x7f15280ab290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ce00640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15280c9110 0x7f15280ab290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.316+0000 7f152ce00640 1 -- 172.21.15.16:0/4223949002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514009390 con 0x7f15281590b0 2024-08-17T02:48:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.317+0000 7f152ce00640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15281590b0 0x7f15281543d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1520001460 tx=0x7f1520001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.317+0000 7f151f400640 1 -- 172.21.15.16:0/4223949002 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152000ce90 con 0x7f15281590b0 2024-08-17T02:48:33.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.317+0000 7f151f400640 1 -- 172.21.15.16:0/4223949002 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152000a850 con 0x7f15281590b0 2024-08-17T02:48:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.317+0000 7f151f400640 1 -- 172.21.15.16:0/4223949002 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152000ac50 con 0x7f15281590b0 2024-08-17T02:48:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.317+0000 7f152ee00640 1 -- 172.21.15.16:0/4223949002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15280ac230 con 0x7f15281590b0 2024-08-17T02:48:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.317+0000 7f152ee00640 1 -- 172.21.15.16:0/4223949002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1528154910 con 0x7f15281590b0 2024-08-17T02:48:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.318+0000 7f152ee00640 1 -- 172.21.15.16:0/4223949002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1528071050 con 0x7f15281590b0 2024-08-17T02:48:33.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.319+0000 7f151f400640 1 -- 172.21.15.16:0/4223949002 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1520007300 con 0x7f15281590b0 2024-08-17T02:48:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.320+0000 7f151f400640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1508065150 0x7f1508067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.320+0000 7f151f400640 1 -- 172.21.15.16:0/4223949002 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f1520082a50 con 0x7f15281590b0 2024-08-17T02:48:33.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.320+0000 7f1527e00640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1508065150 0x7f1508067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:33.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.321+0000 7f1527e00640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1508065150 0x7f1508067600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f15180011e0 tx=0x7f1518009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:33.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.324+0000 7f151f400640 1 -- 172.21.15.16:0/4223949002 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1520050700 con 0x7f15281590b0 2024-08-17T02:48:33.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.507+0000 7f152ee00640 1 -- 172.21.15.16:0/4223949002 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15280c8ef0 con 0x7f15281590b0 2024-08-17T02:48:33.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.507+0000 7f151f400640 1 -- 172.21.15.16:0/4223949002 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1520053d20 con 0x7f15281590b0 2024-08-17T02:48:33.507 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-08-17T02:48:33.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.509+0000 7f152ee00640 1 -- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1508065150 msgr2=0x7f1508067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.509+0000 7f152ee00640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1508065150 0x7f1508067600 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f15180011e0 tx=0x7f1518009040 comp rx=0 tx=0).stop 2024-08-17T02:48:33.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.509+0000 7f152ee00640 1 -- 172.21.15.16:0/4223949002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15281590b0 msgr2=0x7f15281543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.509+0000 7f152ee00640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15281590b0 0x7f15281543d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1520001460 tx=0x7f1520001ae0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.509+0000 7f152ee00640 1 -- 172.21.15.16:0/4223949002 shutdown_connections 2024-08-17T02:48:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.509+0000 7f152ee00640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1508065150 0x7f1508067600 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.509+0000 7f152ee00640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15281590b0 0x7f15281543d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.509+0000 7f152ee00640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1528158ce0 0x7f1528157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.509+0000 7f152ee00640 1 --2- 172.21.15.16:0/4223949002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15280c9110 0x7f15280ab290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.509+0000 7f152ee00640 1 -- 172.21.15.16:0/4223949002 >> 172.21.15.16:0/4223949002 conn(0x7f1528067750 msgr2=0x7f15280c98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.509+0000 7f152ee00640 1 -- 172.21.15.16:0/4223949002 shutdown_connections 2024-08-17T02:48:33.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.509+0000 7f152ee00640 1 -- 172.21.15.16:0/4223949002 wait complete. 2024-08-17T02:48:33.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000521811.meta pg_num 2024-08-17T02:48:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.678+0000 7f494c000640 1 Processor -- start 2024-08-17T02:48:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.678+0000 7f494c000640 1 -- start start 2024-08-17T02:48:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.680+0000 7f494c000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f49440adcf0 0x7f49440ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.680+0000 7f494c000640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49440ab130 con 0x7f4944156c50 2024-08-17T02:48:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.680+0000 7f494c000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49440ae5a0 con 0x7f49440abb00 2024-08-17T02:48:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.680+0000 7f494c000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49440ae2c0 con 0x7f49440adcf0 2024-08-17T02:48:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.680+0000 7f4949600640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4944156c50 0x7f49440c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44374/0 (socket says 172.21.15.16:44374) 2024-08-17T02:48:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.680+0000 7f4949600640 1 -- 172.21.15.16:0/2364164269 learned_addr learned my addr 172.21.15.16:0/2364164269 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:33.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.680+0000 7f4948c00640 1 --2- 172.21.15.16:0/2364164269 >> v2:172.21.15.142:3300/0 conn(0x7f49440adcf0 0x7f49440ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:33.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.680+0000 7f4943e00640 1 -- 172.21.15.16:0/2364164269 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 766495381 0 0) 0x7f49440ab130 con 0x7f4944156c50 2024-08-17T02:48:33.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.680+0000 7f4943e00640 1 -- 172.21.15.16:0/2364164269 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f492c0034e0 con 0x7f4944156c50 2024-08-17T02:48:33.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.681+0000 7f4948c00640 1 -- 172.21.15.16:0/2364164269 >> v1:172.21.15.142:6789/0 conn(0x7f49440abb00 legacy=0x7f49440c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.681+0000 7f4948c00640 1 -- 172.21.15.16:0/2364164269 >> v1:172.21.15.16:6789/0 conn(0x7f4944156c50 legacy=0x7f49440c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.681+0000 7f4943e00640 1 -- 172.21.15.16:0/2364164269 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2363859450 0 0) 0x7f492c0034e0 con 0x7f4944156c50 2024-08-17T02:48:33.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.681+0000 7f4948c00640 1 -- 172.21.15.16:0/2364164269 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440aaae0 con 0x7f49440adcf0 2024-08-17T02:48:33.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.682+0000 7f4948c00640 1 --2- 172.21.15.16:0/2364164269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49440adcf0 0x7f49440ab5c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4930009850 tx=0x7f49300090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b61a1129fa70f4c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:33.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.682+0000 7f4943e00640 1 -- 172.21.15.16:0/2364164269 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493002ad00 con 0x7f49440adcf0 2024-08-17T02:48:33.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.682+0000 7f4943e00640 1 -- 172.21.15.16:0/2364164269 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493002ae60 con 0x7f49440adcf0 2024-08-17T02:48:33.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.682+0000 7f4943e00640 1 -- 172.21.15.16:0/2364164269 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930032670 con 0x7f49440adcf0 2024-08-17T02:48:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.683+0000 7f494c000640 1 -- 172.21.15.16:0/2364164269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49440adcf0 msgr2=0x7f49440ab5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.683+0000 7f494c000640 1 --2- 172.21.15.16:0/2364164269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49440adcf0 0x7f49440ab5c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4930009850 tx=0x7f49300090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.683+0000 7f494c000640 1 -- 172.21.15.16:0/2364164269 shutdown_connections 2024-08-17T02:48:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.683+0000 7f494c000640 1 --2- 172.21.15.16:0/2364164269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49440adcf0 0x7f49440ab5c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.683+0000 7f494c000640 1 -- 172.21.15.16:0/2364164269 >> 172.21.15.16:0/2364164269 conn(0x7f4944067750 msgr2=0x7f49440c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.683+0000 7f494c000640 1 -- 172.21.15.16:0/2364164269 shutdown_connections 2024-08-17T02:48:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.683+0000 7f494c000640 1 -- 172.21.15.16:0/2364164269 wait complete. 2024-08-17T02:48:33.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.684+0000 7f494c000640 1 Processor -- start 2024-08-17T02:48:33.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.684+0000 7f494c000640 1 -- start start 2024-08-17T02:48:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.685+0000 7f494c000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49440abb00 0x7f49441570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.685+0000 7f494c000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49440adcf0 0x7f494416c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.685+0000 7f494c000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944156c50 0x7f494416ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.685+0000 7f494c000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944158530 con 0x7f49440abb00 2024-08-17T02:48:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.685+0000 7f494c000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944157920 con 0x7f4944156c50 2024-08-17T02:48:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.685+0000 7f494c000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4944157c00 con 0x7f49440adcf0 2024-08-17T02:48:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.685+0000 7f4949600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49440abb00 0x7f49441570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.685+0000 7f4949600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49440abb00 0x7f49441570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48270/0 (socket says 172.21.15.16:48270) 2024-08-17T02:48:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.685+0000 7f4949600640 1 -- 172.21.15.16:0/1334136450 learned_addr learned my addr 172.21.15.16:0/1334136450 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:33.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f4948c00640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49440adcf0 0x7f494416c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f494a000640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944156c50 0x7f494416ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f4949600640 1 -- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49440adcf0 msgr2=0x7f494416c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f4949600640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49440adcf0 0x7f494416c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f4949600640 1 -- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944156c50 msgr2=0x7f494416ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f4949600640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944156c50 0x7f494416ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f4949600640 1 -- 172.21.15.16:0/1334136450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930009390 con 0x7f49440abb00 2024-08-17T02:48:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f4948c00640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49440adcf0 0x7f494416c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f4949600640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49440abb00 0x7f49441570d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f49380011e0 tx=0x7f4938007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f4940c00640 1 -- 172.21.15.16:0/1334136450 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938007360 con 0x7f49440abb00 2024-08-17T02:48:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f4940c00640 1 -- 172.21.15.16:0/1334136450 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49380074c0 con 0x7f49440abb00 2024-08-17T02:48:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f4940c00640 1 -- 172.21.15.16:0/1334136450 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380049b0 con 0x7f49440abb00 2024-08-17T02:48:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.686+0000 7f494c000640 1 -- 172.21.15.16:0/1334136450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4944157ee0 con 0x7f49440abb00 2024-08-17T02:48:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.687+0000 7f494c000640 1 -- 172.21.15.16:0/1334136450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4944157610 con 0x7f49440abb00 2024-08-17T02:48:33.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.688+0000 7f494c000640 1 -- 172.21.15.16:0/1334136450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4944071050 con 0x7f49440abb00 2024-08-17T02:48:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.688+0000 7f4940c00640 1 -- 172.21.15.16:0/1334136450 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f4938014420 con 0x7f49440abb00 2024-08-17T02:48:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.690+0000 7f4940c00640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4924065150 0x7f4924067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.690+0000 7f4940c00640 1 -- 172.21.15.16:0/1334136450 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f4938082630 con 0x7f49440abb00 2024-08-17T02:48:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.690+0000 7f4948c00640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4924065150 0x7f4924067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:33.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.691+0000 7f4948c00640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4924065150 0x7f4924067600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4930034000 tx=0x7f4930032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:33.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.694+0000 7f4940c00640 1 -- 172.21.15.16:0/1334136450 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f49380502e0 con 0x7f49440abb00 2024-08-17T02:48:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.876+0000 7f494c000640 1 -- 172.21.15.16:0/1334136450 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000521811.meta", "var": "pg_num"} v 0) v1 -- 0x7f494406a830 con 0x7f49440abb00 2024-08-17T02:48:33.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.876+0000 7f4940c00640 1 -- 172.21.15.16:0/1334136450 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000521811.meta", "var": "pg_num"}]=0 v40) v1 ==== 124+0+11 (secure 0 0 0) 0x7f4938007630 con 0x7f49440abb00 2024-08-17T02:48:33.878 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:48:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.878+0000 7f494c000640 1 -- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4924065150 msgr2=0x7f4924067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.878+0000 7f494c000640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4924065150 0x7f4924067600 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4930034000 tx=0x7f4930032000 comp rx=0 tx=0).stop 2024-08-17T02:48:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.878+0000 7f494c000640 1 -- 172.21.15.16:0/1334136450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49440abb00 msgr2=0x7f49441570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.878+0000 7f494c000640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49440abb00 0x7f49441570d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f49380011e0 tx=0x7f4938007d30 comp rx=0 tx=0).stop 2024-08-17T02:48:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.878+0000 7f494c000640 1 -- 172.21.15.16:0/1334136450 shutdown_connections 2024-08-17T02:48:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.878+0000 7f494c000640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4924065150 0x7f4924067600 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.878+0000 7f494c000640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4944156c50 0x7f494416ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.878+0000 7f494c000640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f49440adcf0 0x7f494416c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.878+0000 7f494c000640 1 --2- 172.21.15.16:0/1334136450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49440abb00 0x7f49441570d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.878+0000 7f494c000640 1 -- 172.21.15.16:0/1334136450 >> 172.21.15.16:0/1334136450 conn(0x7f4944067750 msgr2=0x7f4944153150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:33.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.879+0000 7f494c000640 1 -- 172.21.15.16:0/1334136450 shutdown_connections 2024-08-17T02:48:33.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:33.879+0000 7f494c000640 1 -- 172.21.15.16:0/1334136450 wait complete. 2024-08-17T02:48:33.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000521811.data pg_num 2024-08-17T02:48:34.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.048+0000 7fdf58e00640 1 Processor -- start 2024-08-17T02:48:34.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.048+0000 7fdf58e00640 1 -- start start 2024-08-17T02:48:34.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.049+0000 7fdf58e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf54156c50 0x7fdf540c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.049+0000 7fdf58e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdf540adcf0 0x7fdf540ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.050+0000 7fdf58e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf540ab130 con 0x7fdf540abb00 2024-08-17T02:48:34.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.050+0000 7fdf58e00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf540ae5a0 con 0x7fdf540adcf0 2024-08-17T02:48:34.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.050+0000 7fdf58e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540ae2c0 con 0x7fdf54156c50 2024-08-17T02:48:34.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.050+0000 7fdf4be00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdf540adcf0 0x7fdf540ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.050+0000 7fdf4be00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdf540adcf0 0x7fdf540ab5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41672/0 (socket says 172.21.15.16:41672) 2024-08-17T02:48:34.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.050+0000 7fdf4be00640 1 -- 172.21.15.16:0/3849343231 learned_addr learned my addr 172.21.15.16:0/3849343231 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:34.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.050+0000 7fdf51e00640 1 --2- 172.21.15.16:0/3849343231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf54156c50 0x7fdf540c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.050+0000 7fdf4be00640 1 -- 172.21.15.16:0/3849343231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf54156c50 msgr2=0x7fdf540c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:34.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.050+0000 7fdf4be00640 1 --2- 172.21.15.16:0/3849343231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf54156c50 0x7fdf540c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.050+0000 7fdf4be00640 1 -- 172.21.15.16:0/3849343231 >> v1:172.21.15.142:6789/0 conn(0x7fdf540abb00 legacy=0x7fdf540c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:34.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.050+0000 7fdf4be00640 1 -- 172.21.15.16:0/3849343231 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf540aaae0 con 0x7fdf540adcf0 2024-08-17T02:48:34.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.051+0000 7fdf4be00640 1 --2- 172.21.15.16:0/3849343231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf540adcf0 0x7fdf540ab5c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009910 tx=0x7fdf3c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c8e376da784c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:34.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.052+0000 7fdf51400640 1 -- 172.21.15.16:0/3849343231 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c02ad20 con 0x7fdf540adcf0 2024-08-17T02:48:34.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.052+0000 7fdf51400640 1 -- 172.21.15.16:0/3849343231 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c02ae80 con 0x7fdf540adcf0 2024-08-17T02:48:34.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.052+0000 7fdf51400640 1 -- 172.21.15.16:0/3849343231 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c032670 con 0x7fdf540adcf0 2024-08-17T02:48:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.052+0000 7fdf58e00640 1 -- 172.21.15.16:0/3849343231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf540adcf0 msgr2=0x7fdf540ab5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.052+0000 7fdf58e00640 1 --2- 172.21.15.16:0/3849343231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf540adcf0 0x7fdf540ab5c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009910 tx=0x7fdf3c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.052+0000 7fdf58e00640 1 -- 172.21.15.16:0/3849343231 shutdown_connections 2024-08-17T02:48:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.052+0000 7fdf58e00640 1 --2- 172.21.15.16:0/3849343231 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf540adcf0 0x7fdf540ab5c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.052+0000 7fdf58e00640 1 --2- 172.21.15.16:0/3849343231 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf54156c50 0x7fdf540c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.052+0000 7fdf58e00640 1 -- 172.21.15.16:0/3849343231 >> 172.21.15.16:0/3849343231 conn(0x7fdf54067750 msgr2=0x7fdf540c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.052+0000 7fdf58e00640 1 -- 172.21.15.16:0/3849343231 shutdown_connections 2024-08-17T02:48:34.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.052+0000 7fdf58e00640 1 -- 172.21.15.16:0/3849343231 wait complete. 2024-08-17T02:48:34.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.054+0000 7fdf58e00640 1 Processor -- start 2024-08-17T02:48:34.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.054+0000 7fdf58e00640 1 -- start start 2024-08-17T02:48:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf58e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf540abb00 0x7fdf5414d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf58e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf540adcf0 0x7fdf5414b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf58e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf54156c50 0x7fdf540ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf58e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5414d970 con 0x7fdf540adcf0 2024-08-17T02:48:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf58e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf54158cf0 con 0x7fdf540abb00 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf58e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf54157e30 con 0x7fdf54156c50 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf4be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf540adcf0 0x7fdf5414b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf4be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf540adcf0 0x7fdf5414b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41674/0 (socket says 172.21.15.16:41674) 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf4be00640 1 -- 172.21.15.16:0/449554369 learned_addr learned my addr 172.21.15.16:0/449554369 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf52800640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf54156c50 0x7fdf540ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf51e00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf540abb00 0x7fdf5414d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf4be00640 1 -- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf54156c50 msgr2=0x7fdf540ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf4be00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf54156c50 0x7fdf540ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf4be00640 1 -- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf540abb00 msgr2=0x7fdf5414d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf4be00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf540abb00 0x7fdf5414d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf4be00640 1 -- 172.21.15.16:0/449554369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c009450 con 0x7fdf540adcf0 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.055+0000 7fdf51e00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf540abb00 0x7fdf5414d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.056+0000 7fdf4be00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf540adcf0 0x7fdf5414b400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c02a760 tx=0x7fdf3c032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.056+0000 7fdf49600640 1 -- 172.21.15.16:0/449554369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c03d760 con 0x7fdf540adcf0 2024-08-17T02:48:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.056+0000 7fdf49600640 1 -- 172.21.15.16:0/449554369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c03d8e0 con 0x7fdf540adcf0 2024-08-17T02:48:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.056+0000 7fdf58e00640 1 -- 172.21.15.16:0/449554369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf54158110 con 0x7fdf540adcf0 2024-08-17T02:48:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.056+0000 7fdf49600640 1 -- 172.21.15.16:0/449554369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c03dca0 con 0x7fdf540adcf0 2024-08-17T02:48:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.056+0000 7fdf58e00640 1 -- 172.21.15.16:0/449554369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf54158980 con 0x7fdf540adcf0 2024-08-17T02:48:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.057+0000 7fdf58e00640 1 -- 172.21.15.16:0/449554369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf54158e80 con 0x7fdf540adcf0 2024-08-17T02:48:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.058+0000 7fdf49600640 1 -- 172.21.15.16:0/449554369 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fdf3c039040 con 0x7fdf540adcf0 2024-08-17T02:48:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.059+0000 7fdf49600640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fdf30065150 0x7fdf30067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.059+0000 7fdf49600640 1 -- 172.21.15.16:0/449554369 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7fdf3c0a31e0 con 0x7fdf540adcf0 2024-08-17T02:48:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.059+0000 7fdf51e00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fdf30065150 0x7fdf30067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.060+0000 7fdf51e00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fdf30065150 0x7fdf30067600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdf5414c0c0 tx=0x7fdf44008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:34.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.063+0000 7fdf49600640 1 -- 172.21.15.16:0/449554369 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fdf3c070ec0 con 0x7fdf540adcf0 2024-08-17T02:48:34.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.243+0000 7fdf58e00640 1 -- 172.21.15.16:0/449554369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000521811.data", "var": "pg_num"} v 0) v1 -- 0x7fdf54071050 con 0x7fdf540adcf0 2024-08-17T02:48:34.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.244+0000 7fdf49600640 1 -- 172.21.15.16:0/449554369 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000521811.data", "var": "pg_num"}]=0 v40) v1 ==== 124+0+11 (secure 0 0 0) 0x7fdf3c0744e0 con 0x7fdf540adcf0 2024-08-17T02:48:34.245 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:48:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.246+0000 7fdf58e00640 1 -- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fdf30065150 msgr2=0x7fdf30067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.246+0000 7fdf58e00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fdf30065150 0x7fdf30067600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdf5414c0c0 tx=0x7fdf44008040 comp rx=0 tx=0).stop 2024-08-17T02:48:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.246+0000 7fdf58e00640 1 -- 172.21.15.16:0/449554369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf540adcf0 msgr2=0x7fdf5414b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.246+0000 7fdf58e00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf540adcf0 0x7fdf5414b400 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c02a760 tx=0x7fdf3c032000 comp rx=0 tx=0).stop 2024-08-17T02:48:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.246+0000 7fdf58e00640 1 -- 172.21.15.16:0/449554369 shutdown_connections 2024-08-17T02:48:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.246+0000 7fdf58e00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fdf30065150 0x7fdf30067600 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.246+0000 7fdf58e00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fdf54156c50 0x7fdf540ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.246+0000 7fdf58e00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf540adcf0 0x7fdf5414b400 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.246+0000 7fdf58e00640 1 --2- 172.21.15.16:0/449554369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf540abb00 0x7fdf5414d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.246+0000 7fdf58e00640 1 -- 172.21.15.16:0/449554369 >> 172.21.15.16:0/449554369 conn(0x7fdf54067750 msgr2=0x7fdf5406db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.246+0000 7fdf58e00640 1 -- 172.21.15.16:0/449554369 shutdown_connections 2024-08-17T02:48:34.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.246+0000 7fdf58e00640 1 -- 172.21.15.16:0/449554369 wait complete. 2024-08-17T02:48:34.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:48:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.411+0000 7f315b000640 1 Processor -- start 2024-08-17T02:48:34.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.412+0000 7f315b000640 1 -- start start 2024-08-17T02:48:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.413+0000 7f315b000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f31540ceb40 0x7f31540cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.413+0000 7f315b000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31540ccad0 0x7f3154162860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.413+0000 7f315b000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31540ccf30 con 0x7f3154159ca0 2024-08-17T02:48:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.413+0000 7f315b000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31540cd6e0 con 0x7f31540ccad0 2024-08-17T02:48:34.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.413+0000 7f315b000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31540cadc0 con 0x7f31540ceb40 2024-08-17T02:48:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.413+0000 7f315a600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31540ccad0 0x7f3154162860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.413+0000 7f315a600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31540ccad0 0x7f3154162860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41688/0 (socket says 172.21.15.16:41688) 2024-08-17T02:48:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.413+0000 7f315a600640 1 -- 172.21.15.16:0/1904114631 learned_addr learned my addr 172.21.15.16:0/1904114631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:34.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.413+0000 7f3159200640 1 --2- 172.21.15.16:0/1904114631 >> v2:172.21.15.142:3300/0 conn(0x7f31540ceb40 0x7f31540cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.414+0000 7f315a600640 1 -- 172.21.15.16:0/1904114631 >> v2:172.21.15.142:3300/0 conn(0x7f31540ceb40 msgr2=0x7f31540cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.414+0000 7f315a600640 1 --2- 172.21.15.16:0/1904114631 >> v2:172.21.15.142:3300/0 conn(0x7f31540ceb40 0x7f31540cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.414+0000 7f315a600640 1 -- 172.21.15.16:0/1904114631 >> v1:172.21.15.142:6789/0 conn(0x7f3154159ca0 legacy=0x7f31540ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.414+0000 7f315a600640 1 -- 172.21.15.16:0/1904114631 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31540cb060 con 0x7f31540ccad0 2024-08-17T02:48:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.415+0000 7f315a600640 1 --2- 172.21.15.16:0/1904114631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31540ccad0 0x7f3154162860 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f315408a060 tx=0x7f314c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bac509ccd11196d server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.415+0000 7f3153e00640 1 -- 172.21.15.16:0/1904114631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c032400 con 0x7f31540ccad0 2024-08-17T02:48:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.415+0000 7f3153e00640 1 -- 172.21.15.16:0/1904114631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c02e050 con 0x7f31540ccad0 2024-08-17T02:48:34.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.415+0000 7f3153e00640 1 -- 172.21.15.16:0/1904114631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c0327f0 con 0x7f31540ccad0 2024-08-17T02:48:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.415+0000 7f315b000640 1 -- 172.21.15.16:0/1904114631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31540ccad0 msgr2=0x7f3154162860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.415+0000 7f315b000640 1 --2- 172.21.15.16:0/1904114631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31540ccad0 0x7f3154162860 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f315408a060 tx=0x7f314c009840 comp rx=0 tx=0).stop 2024-08-17T02:48:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.415+0000 7f315b000640 1 -- 172.21.15.16:0/1904114631 shutdown_connections 2024-08-17T02:48:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.415+0000 7f315b000640 1 --2- 172.21.15.16:0/1904114631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31540ccad0 0x7f3154162860 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.415+0000 7f315b000640 1 --2- 172.21.15.16:0/1904114631 >> v2:172.21.15.142:3300/0 conn(0x7f31540ceb40 0x7f31540cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.415+0000 7f315b000640 1 -- 172.21.15.16:0/1904114631 >> 172.21.15.16:0/1904114631 conn(0x7f3154098e00 msgr2=0x7f315406eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.416+0000 7f315b000640 1 -- 172.21.15.16:0/1904114631 shutdown_connections 2024-08-17T02:48:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.416+0000 7f315b000640 1 -- 172.21.15.16:0/1904114631 wait complete. 2024-08-17T02:48:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.417+0000 7f315b000640 1 Processor -- start 2024-08-17T02:48:34.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.417+0000 7f315b000640 1 -- start start 2024-08-17T02:48:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.418+0000 7f315b000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31540ccad0 0x7f315416bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.418+0000 7f315b000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31540ceb40 0x7f31540c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.418+0000 7f315b000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3154159ca0 0x7f3154169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.418+0000 7f315b000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3154155990 con 0x7f31540ceb40 2024-08-17T02:48:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.418+0000 7f315b000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31540c6bc0 con 0x7f3154159ca0 2024-08-17T02:48:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.418+0000 7f315b000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31540c7740 con 0x7f31540ccad0 2024-08-17T02:48:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.418+0000 7f3159200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31540ceb40 0x7f31540c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.418+0000 7f3159200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31540ceb40 0x7f31540c5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41694/0 (socket says 172.21.15.16:41694) 2024-08-17T02:48:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.418+0000 7f3159200640 1 -- 172.21.15.16:0/2069733046 learned_addr learned my addr 172.21.15.16:0/2069733046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.418+0000 7f315a600640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3154159ca0 0x7f3154169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.418+0000 7f3159c00640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31540ccad0 0x7f315416bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.419+0000 7f3159200640 1 -- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31540ccad0 msgr2=0x7f315416bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.419+0000 7f3159200640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31540ccad0 0x7f315416bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.419+0000 7f3159200640 1 -- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3154159ca0 msgr2=0x7f3154169f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.419+0000 7f3159200640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3154159ca0 0x7f3154169f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.419+0000 7f3159200640 1 -- 172.21.15.16:0/2069733046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c009ae0 con 0x7f31540ceb40 2024-08-17T02:48:34.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.419+0000 7f3159200640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31540ceb40 0x7f31540c5600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f314000ea20 tx=0x7f31400090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.419+0000 7f3150c00640 1 -- 172.21.15.16:0/2069733046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314000ca10 con 0x7f31540ceb40 2024-08-17T02:48:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.419+0000 7f3150c00640 1 -- 172.21.15.16:0/2069733046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314000cb70 con 0x7f31540ceb40 2024-08-17T02:48:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.419+0000 7f3150c00640 1 -- 172.21.15.16:0/2069733046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3140013470 con 0x7f31540ceb40 2024-08-17T02:48:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.419+0000 7f315b000640 1 -- 172.21.15.16:0/2069733046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31540c5b40 con 0x7f31540ceb40 2024-08-17T02:48:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.419+0000 7f315b000640 1 -- 172.21.15.16:0/2069733046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31540c6e10 con 0x7f31540ceb40 2024-08-17T02:48:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.420+0000 7f315b000640 1 -- 172.21.15.16:0/2069733046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f315406eb90 con 0x7f31540ceb40 2024-08-17T02:48:34.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.421+0000 7f3150c00640 1 -- 172.21.15.16:0/2069733046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f3140013860 con 0x7f31540ceb40 2024-08-17T02:48:34.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.422+0000 7f3150c00640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3134065150 0x7f3134067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.422+0000 7f3150c00640 1 -- 172.21.15.16:0/2069733046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f3140082bc0 con 0x7f31540ceb40 2024-08-17T02:48:34.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.422+0000 7f3159c00640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3134065150 0x7f3134067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.423+0000 7f3159c00640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3134065150 0x7f3134067600 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f31540b5010 tx=0x7f3148007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:34.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.426+0000 7f3150c00640 1 -- 172.21.15.16:0/2069733046 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f315406eb90 con 0x7f31540ceb40 2024-08-17T02:48:34.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.647+0000 7f315b000640 1 -- 172.21.15.16:0/2069733046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f315406eb90 con 0x7f31540ceb40 2024-08-17T02:48:34.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.648+0000 7f3150c00640 1 -- 172.21.15.16:0/2069733046 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5687 (secure 0 0 0) 0x7f315406eb90 con 0x7f31540ceb40 2024-08-17T02:48:34.649 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:34.649 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2889208700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2889208700},{"type":"v1","addr":"172.21.15.142:6837","nonce":2889208700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4287834697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4287834697},{"type":"v1","addr":"172.21.15.142:6835","nonce":4287834697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:16.190199+0000","modified":"2024-08-17T02:46:19.780153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6835/219684854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":219684854},{"type":"v1","addr":"172.21.15.16:6835","nonce":219684854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:28.938945+0000","modified":"2024-08-17T02:46:30.943869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6834/3115083427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3115083427},{"type":"v1","addr":"172.21.15.16:6834","nonce":3115083427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000521811","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:48:34.649 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2024-08-17T02:48:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.650+0000 7f315b000640 1 -- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3134065150 msgr2=0x7f3134067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.650+0000 7f315b000640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3134065150 0x7f3134067600 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f31540b5010 tx=0x7f3148007e30 comp rx=0 tx=0).stop 2024-08-17T02:48:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.650+0000 7f315b000640 1 -- 172.21.15.16:0/2069733046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31540ceb40 msgr2=0x7f31540c5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.650+0000 7f315b000640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31540ceb40 0x7f31540c5600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f314000ea20 tx=0x7f31400090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.650+0000 7f315b000640 1 -- 172.21.15.16:0/2069733046 shutdown_connections 2024-08-17T02:48:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.650+0000 7f315b000640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3134065150 0x7f3134067600 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.650+0000 7f315b000640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3154159ca0 0x7f3154169f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.650+0000 7f315b000640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31540ceb40 0x7f31540c5600 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.650+0000 7f315b000640 1 --2- 172.21.15.16:0/2069733046 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f31540ccad0 0x7f315416bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.650+0000 7f315b000640 1 -- 172.21.15.16:0/2069733046 >> 172.21.15.16:0/2069733046 conn(0x7f3154098e00 msgr2=0x7f3154156d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:34.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.650+0000 7f315b000640 1 -- 172.21.15.16:0/2069733046 shutdown_connections 2024-08-17T02:48:34.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.650+0000 7f315b000640 1 -- 172.21.15.16:0/2069733046 wait complete. 2024-08-17T02:48:34.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:48:34.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.817+0000 7f487a600640 1 Processor -- start 2024-08-17T02:48:34.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.817+0000 7f487a600640 1 -- start start 2024-08-17T02:48:34.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.818+0000 7f487a600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f48740b1440 0x7f48740caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.818+0000 7f487a600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4874140580 0x7f48740c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.818+0000 7f487a600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48740caa80 con 0x7f48741401b0 2024-08-17T02:48:34.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.818+0000 7f487a600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48740cb450 con 0x7f4874140580 2024-08-17T02:48:34.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.818+0000 7f487a600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48740c9e70 con 0x7f48740b1440 2024-08-17T02:48:34.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.819+0000 7f4873e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4874140580 0x7f48740c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.819+0000 7f4873e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4874140580 0x7f48740c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41702/0 (socket says 172.21.15.16:41702) 2024-08-17T02:48:34.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.819+0000 7f4873e00640 1 -- 172.21.15.16:0/470388208 learned_addr learned my addr 172.21.15.16:0/470388208 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:34.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.819+0000 7f4872a00640 1 --2- 172.21.15.16:0/470388208 >> v2:172.21.15.142:3300/0 conn(0x7f48740b1440 0x7f48740caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.819+0000 7f4873e00640 1 -- 172.21.15.16:0/470388208 >> v2:172.21.15.142:3300/0 conn(0x7f48740b1440 msgr2=0x7f48740caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:34.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.819+0000 7f4873e00640 1 --2- 172.21.15.16:0/470388208 >> v2:172.21.15.142:3300/0 conn(0x7f48740b1440 0x7f48740caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.819+0000 7f4873e00640 1 -- 172.21.15.16:0/470388208 >> v1:172.21.15.142:6789/0 conn(0x7f48741401b0 legacy=0x7f48740b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:34.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.819+0000 7f4873e00640 1 -- 172.21.15.16:0/470388208 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48740ca150 con 0x7f4874140580 2024-08-17T02:48:34.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.820+0000 7f4873e00640 1 --2- 172.21.15.16:0/470388208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4874140580 0x7f48740c9040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f487406aa80 tx=0x7f486c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75dd30e241f5ca1e server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:34.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.821+0000 7f4872000640 1 -- 172.21.15.16:0/470388208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c02e050 con 0x7f4874140580 2024-08-17T02:48:34.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.821+0000 7f4872000640 1 -- 172.21.15.16:0/470388208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c007400 con 0x7f4874140580 2024-08-17T02:48:34.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.821+0000 7f4872000640 1 -- 172.21.15.16:0/470388208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c0077e0 con 0x7f4874140580 2024-08-17T02:48:34.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.821+0000 7f487a600640 1 -- 172.21.15.16:0/470388208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4874140580 msgr2=0x7f48740c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.821+0000 7f487a600640 1 --2- 172.21.15.16:0/470388208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4874140580 0x7f48740c9040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f487406aa80 tx=0x7f486c009840 comp rx=0 tx=0).stop 2024-08-17T02:48:34.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.821+0000 7f487a600640 1 -- 172.21.15.16:0/470388208 shutdown_connections 2024-08-17T02:48:34.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.821+0000 7f487a600640 1 --2- 172.21.15.16:0/470388208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4874140580 0x7f48740c9040 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.821+0000 7f487a600640 1 --2- 172.21.15.16:0/470388208 >> v2:172.21.15.142:3300/0 conn(0x7f48740b1440 0x7f48740caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.821+0000 7f487a600640 1 -- 172.21.15.16:0/470388208 >> 172.21.15.16:0/470388208 conn(0x7f48740cb9b0 msgr2=0x7f48740cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:34.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.821+0000 7f487a600640 1 -- 172.21.15.16:0/470388208 shutdown_connections 2024-08-17T02:48:34.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.821+0000 7f487a600640 1 -- 172.21.15.16:0/470388208 wait complete. 2024-08-17T02:48:34.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.823+0000 7f487a600640 1 Processor -- start 2024-08-17T02:48:34.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.823+0000 7f487a600640 1 -- start start 2024-08-17T02:48:34.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.824+0000 7f487a600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48740b1440 0x7f4874137b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.824+0000 7f487a600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48741401b0 0x7f487415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.824+0000 7f487a600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4874140580 0x7f4874156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.824+0000 7f487a600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4874138d80 con 0x7f48741401b0 2024-08-17T02:48:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.824+0000 7f487a600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487415baf0 con 0x7f48740b1440 2024-08-17T02:48:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.824+0000 7f487a600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4874138170 con 0x7f4874140580 2024-08-17T02:48:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.824+0000 7f4872a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48741401b0 0x7f487415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.824+0000 7f4872a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48741401b0 0x7f487415b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41716/0 (socket says 172.21.15.16:41716) 2024-08-17T02:48:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.824+0000 7f4872a00640 1 -- 172.21.15.16:0/4094244329 learned_addr learned my addr 172.21.15.16:0/4094244329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.825+0000 7f4872a00640 1 -- 172.21.15.16:0/4094244329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4874140580 msgr2=0x7f4874156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.825+0000 7f4872a00640 1 --2- 172.21.15.16:0/4094244329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4874140580 0x7f4874156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.825+0000 7f4872a00640 1 -- 172.21.15.16:0/4094244329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48740b1440 msgr2=0x7f4874137b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.825+0000 7f4872a00640 1 --2- 172.21.15.16:0/4094244329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48740b1440 0x7f4874137b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:34.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.825+0000 7f4872a00640 1 -- 172.21.15.16:0/4094244329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c009ae0 con 0x7f48741401b0 2024-08-17T02:48:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.825+0000 7f4872a00640 1 --2- 172.21.15.16:0/4094244329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48741401b0 0x7f487415b5b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f486000d890 tx=0x7f4860008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:34.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.825+0000 7f486aa00640 1 -- 172.21.15.16:0/4094244329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486000de20 con 0x7f48741401b0 2024-08-17T02:48:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.825+0000 7f486aa00640 1 -- 172.21.15.16:0/4094244329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48600098b0 con 0x7f48741401b0 2024-08-17T02:48:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.825+0000 7f486aa00640 1 -- 172.21.15.16:0/4094244329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860009c70 con 0x7f48741401b0 2024-08-17T02:48:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.825+0000 7f487a600640 1 -- 172.21.15.16:0/4094244329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4874138450 con 0x7f48741401b0 2024-08-17T02:48:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.825+0000 7f487a600640 1 -- 172.21.15.16:0/4094244329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4874138a10 con 0x7f48741401b0 2024-08-17T02:48:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.826+0000 7f487a600640 1 -- 172.21.15.16:0/4094244329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4874064db0 con 0x7f48741401b0 2024-08-17T02:48:34.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.827+0000 7f486aa00640 1 -- 172.21.15.16:0/4094244329 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f4860014410 con 0x7f48741401b0 2024-08-17T02:48:34.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.828+0000 7f486aa00640 1 --2- 172.21.15.16:0/4094244329 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4854065150 0x7f4854067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:34.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.828+0000 7f486aa00640 1 -- 172.21.15.16:0/4094244329 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f48600831b0 con 0x7f48741401b0 2024-08-17T02:48:34.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.828+0000 7f4873400640 1 --2- 172.21.15.16:0/4094244329 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4854065150 0x7f4854067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:34.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.829+0000 7f4873400640 1 --2- 172.21.15.16:0/4094244329 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4854065150 0x7f4854067600 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f48640016a0 tx=0x7f4864007ab0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:34.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:34.832+0000 7f486aa00640 1 -- 172.21.15.16:0/4094244329 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4860050ea0 con 0x7f48741401b0 2024-08-17T02:48:35.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.016+0000 7f487a600640 1 -- 172.21.15.16:0/4094244329 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f487406a210 con 0x7f48741401b0 2024-08-17T02:48:35.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.017+0000 7f486aa00640 1 -- 172.21.15.16:0/4094244329 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18788 (secure 0 0 0) 0x7f48600544c0 con 0x7f48741401b0 2024-08-17T02:48:35.019 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:35.019 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:47:07.371280+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:46:14.322032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:46:15.170652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000521811.meta","create_time":"2024-08-17T02:46:27.102528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000521811"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000521811.data","create_time":"2024-08-17T02:46:27.975177+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000521811"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.e","mappings":[{"from":4,"to":5}]},{"pgid":"5.27","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]},{"pgid":"5.37","mappings":[{"from":4,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:48:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.019+0000 7f487a600640 1 -- 172.21.15.16:0/4094244329 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4854065150 msgr2=0x7f4854067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.019+0000 7f487a600640 1 --2- 172.21.15.16:0/4094244329 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4854065150 0x7f4854067600 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f48640016a0 tx=0x7f4864007ab0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.019+0000 7f487a600640 1 -- 172.21.15.16:0/4094244329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48741401b0 msgr2=0x7f487415b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.019+0000 7f487a600640 1 --2- 172.21.15.16:0/4094244329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48741401b0 0x7f487415b5b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f486000d890 tx=0x7f4860008040 comp rx=0 tx=0).stop 2024-08-17T02:48:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.020+0000 7f487a600640 1 -- 172.21.15.16:0/4094244329 shutdown_connections 2024-08-17T02:48:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.020+0000 7f487a600640 1 --2- 172.21.15.16:0/4094244329 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4854065150 0x7f4854067600 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.020+0000 7f487a600640 1 --2- 172.21.15.16:0/4094244329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4874140580 0x7f4874156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.020+0000 7f487a600640 1 --2- 172.21.15.16:0/4094244329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48741401b0 0x7f487415b5b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.020+0000 7f487a600640 1 --2- 172.21.15.16:0/4094244329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48740b1440 0x7f4874137b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.020+0000 7f487a600640 1 -- 172.21.15.16:0/4094244329 >> 172.21.15.16:0/4094244329 conn(0x7f48740cb9b0 msgr2=0x7f487406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:35.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.020+0000 7f487a600640 1 -- 172.21.15.16:0/4094244329 shutdown_connections 2024-08-17T02:48:35.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.020+0000 7f487a600640 1 -- 172.21.15.16:0/4094244329 wait complete. 2024-08-17T02:48:35.036 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-17T02:48:35.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:48:35.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.187+0000 7f4eaaa00640 1 Processor -- start 2024-08-17T02:48:35.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.187+0000 7f4eaaa00640 1 -- start start 2024-08-17T02:48:35.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.188+0000 7f4eaaa00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ea4155a20 0x7f4ea40adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.188+0000 7f4eaaa00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea4156670 con 0x7f4ea4155df0 2024-08-17T02:48:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.188+0000 7f4eaaa00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea406e5e0 con 0x7f4ea406df00 2024-08-17T02:48:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.188+0000 7f4eaaa00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea40b0350 con 0x7f4ea4155a20 2024-08-17T02:48:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.188+0000 7f4ea3e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ea4155a20 0x7f4ea40adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.189+0000 7f4ea3e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ea4155a20 0x7f4ea40adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41732/0 (socket says 172.21.15.16:41732) 2024-08-17T02:48:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.189+0000 7f4ea3e00640 1 -- 172.21.15.16:0/1306079470 learned_addr learned my addr 172.21.15.16:0/1306079470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.189+0000 7f4ea3e00640 1 -- 172.21.15.16:0/1306079470 >> v1:172.21.15.142:6789/0 conn(0x7f4ea406df00 legacy=0x7f4ea4156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.189+0000 7f4ea3e00640 1 -- 172.21.15.16:0/1306079470 >> v1:172.21.15.16:6789/0 conn(0x7f4ea4155df0 legacy=0x7f4ea406d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.189+0000 7f4ea3e00640 1 -- 172.21.15.16:0/1306079470 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea406e8c0 con 0x7f4ea4155a20 2024-08-17T02:48:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.190+0000 7f4ea3e00640 1 --2- 172.21.15.16:0/1306079470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea4155a20 0x7f4ea40adf10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009910 tx=0x7f4e940091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c45575e27c0fb78 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.190+0000 7f4ea2a00640 1 -- 172.21.15.16:0/1306079470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9402ae60 con 0x7f4ea4155a20 2024-08-17T02:48:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.190+0000 7f4ea2a00640 1 -- 172.21.15.16:0/1306079470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e94032440 con 0x7f4ea4155a20 2024-08-17T02:48:35.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.190+0000 7f4ea2a00640 1 -- 172.21.15.16:0/1306079470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e940327e0 con 0x7f4ea4155a20 2024-08-17T02:48:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.191+0000 7f4eaaa00640 1 -- 172.21.15.16:0/1306079470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea4155a20 msgr2=0x7f4ea40adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.191+0000 7f4eaaa00640 1 --2- 172.21.15.16:0/1306079470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea4155a20 0x7f4ea40adf10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009910 tx=0x7f4e940091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.191+0000 7f4eaaa00640 1 -- 172.21.15.16:0/1306079470 shutdown_connections 2024-08-17T02:48:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.191+0000 7f4eaaa00640 1 --2- 172.21.15.16:0/1306079470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea4155a20 0x7f4ea40adf10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.191+0000 7f4eaaa00640 1 -- 172.21.15.16:0/1306079470 >> 172.21.15.16:0/1306079470 conn(0x7f4ea40cb9b0 msgr2=0x7f4ea40cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.191+0000 7f4eaaa00640 1 -- 172.21.15.16:0/1306079470 shutdown_connections 2024-08-17T02:48:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.191+0000 7f4eaaa00640 1 -- 172.21.15.16:0/1306079470 wait complete. 2024-08-17T02:48:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.192+0000 7f4eaaa00640 1 Processor -- start 2024-08-17T02:48:35.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.193+0000 7f4eaaa00640 1 -- start start 2024-08-17T02:48:35.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.193+0000 7f4eaaa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea406df00 0x7f4ea414f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.193+0000 7f4eaaa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea4155a20 0x7f4ea4069030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.193+0000 7f4eaaa00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ea4155df0 0x7f4ea4069570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.193+0000 7f4eaaa00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea415d490 con 0x7f4ea406df00 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.193+0000 7f4eaaa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea415c8c0 con 0x7f4ea4155a20 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.193+0000 7f4eaaa00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea415cb60 con 0x7f4ea4155df0 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea406df00 0x7f4ea414f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea406df00 0x7f4ea414f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41740/0 (socket says 172.21.15.16:41740) 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea3e00640 1 -- 172.21.15.16:0/875289436 learned_addr learned my addr 172.21.15.16:0/875289436 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea3400640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea4155a20 0x7f4ea4069030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea8a00640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ea4155df0 0x7f4ea4069570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea3e00640 1 -- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ea4155df0 msgr2=0x7f4ea4069570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea3e00640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ea4155df0 0x7f4ea4069570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea3e00640 1 -- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea4155a20 msgr2=0x7f4ea4069030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea3e00640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea4155a20 0x7f4ea4069030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea3e00640 1 -- 172.21.15.16:0/875289436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e94009450 con 0x7f4ea406df00 2024-08-17T02:48:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea3400640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea4155a20 0x7f4ea4069030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea8a00640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ea4155df0 0x7f4ea4069570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.194+0000 7f4ea3e00640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea406df00 0x7f4ea414f450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4e94035040 tx=0x7f4e94032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.195+0000 7f4e9b400640 1 -- 172.21.15.16:0/875289436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94031de0 con 0x7f4ea406df00 2024-08-17T02:48:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.195+0000 7f4e9b400640 1 -- 172.21.15.16:0/875289436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9403d420 con 0x7f4ea406df00 2024-08-17T02:48:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.195+0000 7f4eaaa00640 1 -- 172.21.15.16:0/875289436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea415ce40 con 0x7f4ea406df00 2024-08-17T02:48:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.195+0000 7f4e9b400640 1 -- 172.21.15.16:0/875289436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9403d820 con 0x7f4ea406df00 2024-08-17T02:48:35.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.195+0000 7f4eaaa00640 1 -- 172.21.15.16:0/875289436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea414f990 con 0x7f4ea406df00 2024-08-17T02:48:35.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.196+0000 7f4eaaa00640 1 -- 172.21.15.16:0/875289436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea4064db0 con 0x7f4ea406df00 2024-08-17T02:48:35.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.197+0000 7f4e9b400640 1 -- 172.21.15.16:0/875289436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f4e94039040 con 0x7f4ea406df00 2024-08-17T02:48:35.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.198+0000 7f4e9b400640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4e84065150 0x7f4e84067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.198+0000 7f4e9b400640 1 -- 172.21.15.16:0/875289436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f4e940a2d60 con 0x7f4ea406df00 2024-08-17T02:48:35.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.198+0000 7f4ea3400640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4e84065150 0x7f4e84067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.199+0000 7f4ea3400640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4e84065150 0x7f4e84067600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4ea416c530 tx=0x7f4e90009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:35.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.202+0000 7f4e9b400640 1 -- 172.21.15.16:0/875289436 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4e94070a30 con 0x7f4ea406df00 2024-08-17T02:48:35.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.424+0000 7f4eaaa00640 1 -- 172.21.15.16:0/875289436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ea415f7b0 con 0x7f4ea406df00 2024-08-17T02:48:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.425+0000 7f4e9b400640 1 -- 172.21.15.16:0/875289436 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5687 (secure 0 0 0) 0x7f4e9402ae60 con 0x7f4ea406df00 2024-08-17T02:48:35.426 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:35.426 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2889208700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2889208700},{"type":"v1","addr":"172.21.15.142:6837","nonce":2889208700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4287834697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4287834697},{"type":"v1","addr":"172.21.15.142:6835","nonce":4287834697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:16.190199+0000","modified":"2024-08-17T02:46:19.780153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6835/219684854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":219684854},{"type":"v1","addr":"172.21.15.16:6835","nonce":219684854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:28.938945+0000","modified":"2024-08-17T02:46:30.943869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6834/3115083427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3115083427},{"type":"v1","addr":"172.21.15.16:6834","nonce":3115083427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000521811","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:48:35.426 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2024-08-17T02:48:35.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.427+0000 7f4eaaa00640 1 -- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4e84065150 msgr2=0x7f4e84067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.427+0000 7f4eaaa00640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4e84065150 0x7f4e84067600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4ea416c530 tx=0x7f4e90009040 comp rx=0 tx=0).stop 2024-08-17T02:48:35.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.427+0000 7f4eaaa00640 1 -- 172.21.15.16:0/875289436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea406df00 msgr2=0x7f4ea414f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.427+0000 7f4eaaa00640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea406df00 0x7f4ea414f450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4e94035040 tx=0x7f4e94032000 comp rx=0 tx=0).stop 2024-08-17T02:48:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.427+0000 7f4eaaa00640 1 -- 172.21.15.16:0/875289436 shutdown_connections 2024-08-17T02:48:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.427+0000 7f4eaaa00640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4e84065150 0x7f4e84067600 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.427+0000 7f4eaaa00640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ea4155df0 0x7f4ea4069570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.427+0000 7f4eaaa00640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea4155a20 0x7f4ea4069030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.427+0000 7f4eaaa00640 1 --2- 172.21.15.16:0/875289436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ea406df00 0x7f4ea414f450 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.427+0000 7f4eaaa00640 1 -- 172.21.15.16:0/875289436 >> 172.21.15.16:0/875289436 conn(0x7f4ea40cb9b0 msgr2=0x7f4ea4152340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.427+0000 7f4eaaa00640 1 -- 172.21.15.16:0/875289436 shutdown_connections 2024-08-17T02:48:35.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.427+0000 7f4eaaa00640 1 -- 172.21.15.16:0/875289436 wait complete. 2024-08-17T02:48:35.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:48:35.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.594+0000 7f3ed9800640 1 Processor -- start 2024-08-17T02:48:35.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.594+0000 7f3ed9800640 1 -- start start 2024-08-17T02:48:35.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.595+0000 7f3ed9800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3ed41401b0 0x7f3ed40caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.595+0000 7f3ed9800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed40caa80 con 0x7f3ed40b1440 2024-08-17T02:48:35.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.595+0000 7f3ed9800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed40c9e70 con 0x7f3ed4140580 2024-08-17T02:48:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.595+0000 7f3ed9800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed40cb450 con 0x7f3ed41401b0 2024-08-17T02:48:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.595+0000 7f3ed3400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3ed40b1440 0x7f3ed40c9040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45186/0 (socket says 172.21.15.16:45186) 2024-08-17T02:48:35.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed3400640 1 -- 172.21.15.16:0/3301230848 learned_addr learned my addr 172.21.15.16:0/3301230848 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed2a00640 1 --2- 172.21.15.16:0/3301230848 >> v2:172.21.15.142:3300/0 conn(0x7f3ed41401b0 0x7f3ed40caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856165214 0 0) 0x7f3ed40caa80 con 0x7f3ed40b1440 2024-08-17T02:48:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3eb80034e0 con 0x7f3ed40b1440 2024-08-17T02:48:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396399245 0 0) 0x7f3ed40c9e70 con 0x7f3ed4140580 2024-08-17T02:48:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed40caa80 con 0x7f3ed4140580 2024-08-17T02:48:35.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1544747616 0 0) 0x7f3eb80034e0 con 0x7f3ed40b1440 2024-08-17T02:48:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 >> v2:172.21.15.142:3300/0 conn(0x7f3ed41401b0 msgr2=0x7f3ed40caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 --2- 172.21.15.16:0/3301230848 >> v2:172.21.15.142:3300/0 conn(0x7f3ed41401b0 0x7f3ed40caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 >> v1:172.21.15.142:6789/0 conn(0x7f3ed4140580 legacy=0x7f3ed40b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed40ca150 con 0x7f3ed40b1440 2024-08-17T02:48:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7f3ecc0017f0 con 0x7f3ed40b1440 2024-08-17T02:48:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 >> v1:172.21.15.16:6789/0 conn(0x7f3ed40b1440 legacy=0x7f3ed40c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 --2- 172.21.15.16:0/3301230848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb8003b60 0x7f3eb8003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 --2- 172.21.15.16:0/3301230848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb8004780 0x7f3eb8004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.596+0000 7f3ed1600640 1 --2- 172.21.15.16:0/3301230848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eb8005130 0x7f3eb800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40cb450 con 0x7f3eb8003b60 2024-08-17T02:48:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb800c0d0 con 0x7f3eb8005130 2024-08-17T02:48:35.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb800c290 con 0x7f3eb8004780 2024-08-17T02:48:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed2a00640 1 --2- 172.21.15.16:0/3301230848 >> v2:172.21.15.142:3300/0 conn(0x7f3ed41401b0 0x7f3ed40caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:48:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed2a00640 1 --2- 172.21.15.16:0/3301230848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb8003b60 0x7f3eb8003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed3400640 1 --2- 172.21.15.16:0/3301230848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eb8005130 0x7f3eb800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed2000640 1 --2- 172.21.15.16:0/3301230848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb8004780 0x7f3eb8004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed2a00640 1 -- 172.21.15.16:0/3301230848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb8004780 msgr2=0x7f3eb8004bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed2a00640 1 --2- 172.21.15.16:0/3301230848 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb8004780 0x7f3eb8004bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed2a00640 1 -- 172.21.15.16:0/3301230848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eb8005130 msgr2=0x7f3eb800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed2a00640 1 --2- 172.21.15.16:0/3301230848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eb8005130 0x7f3eb800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed2a00640 1 -- 172.21.15.16:0/3301230848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb80034e0 con 0x7f3eb8003b60 2024-08-17T02:48:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.597+0000 7f3ed3400640 1 -- 172.21.15.16:0/3301230848 reap_dead start 2024-08-17T02:48:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.598+0000 7f3ed2a00640 1 --2- 172.21.15.16:0/3301230848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb8003b60 0x7f3eb8003f50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4006e70 tx=0x7f3ec40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746c146275ae0ba3 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.599+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4031400 con 0x7f3eb8003b60 2024-08-17T02:48:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.599+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec402d050 con 0x7f3eb8003b60 2024-08-17T02:48:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.599+0000 7f3ed1600640 1 -- 172.21.15.16:0/3301230848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4031790 con 0x7f3eb8003b60 2024-08-17T02:48:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.599+0000 7f3ed9800640 1 -- 172.21.15.16:0/3301230848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb8003b60 msgr2=0x7f3eb8003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.599+0000 7f3ed9800640 1 --2- 172.21.15.16:0/3301230848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb8003b60 0x7f3eb8003f50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4006e70 tx=0x7f3ec40090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.599+0000 7f3ed9800640 1 -- 172.21.15.16:0/3301230848 shutdown_connections 2024-08-17T02:48:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.599+0000 7f3ed9800640 1 --2- 172.21.15.16:0/3301230848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3eb8003b60 0x7f3eb8003f50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.599+0000 7f3ed9800640 1 -- 172.21.15.16:0/3301230848 >> 172.21.15.16:0/3301230848 conn(0x7f3ed40cb9b0 msgr2=0x7f3ed40cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.599+0000 7f3ed9800640 1 -- 172.21.15.16:0/3301230848 shutdown_connections 2024-08-17T02:48:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.599+0000 7f3ed9800640 1 -- 172.21.15.16:0/3301230848 wait complete. 2024-08-17T02:48:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.601+0000 7f3ed9800640 1 Processor -- start 2024-08-17T02:48:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.601+0000 7f3ed9800640 1 -- start start 2024-08-17T02:48:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed9800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb8003b60 0x7f3ed41564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed9800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed415a2d0 0x7f3ed415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed40a9920 0x7f3ed40a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed9800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed415bda0 con 0x7f3ed415a2d0 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed9800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed415b910 con 0x7f3ed40a9920 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed9800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40aa2b0 con 0x7f3eb8003b60 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed415a2d0 0x7f3ed415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed2000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed415a2d0 0x7f3ed415a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41756/0 (socket says 172.21.15.16:41756) 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed2000640 1 -- 172.21.15.16:0/189736860 learned_addr learned my addr 172.21.15.16:0/189736860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed2a00640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb8003b60 0x7f3ed41564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed3400640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed40a9920 0x7f3ed40a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed2000640 1 -- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb8003b60 msgr2=0x7f3ed41564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed2000640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb8003b60 0x7f3ed41564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed2000640 1 -- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed40a9920 msgr2=0x7f3ed40a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed2000640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed40a9920 0x7f3ed40a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed2000640 1 -- 172.21.15.16:0/189736860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed415ad00 con 0x7f3ed415a2d0 2024-08-17T02:48:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed2a00640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb8003b60 0x7f3ed41564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.602+0000 7f3ed3400640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed40a9920 0x7f3ed40a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.603+0000 7f3ed2000640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed415a2d0 0x7f3ed415a720 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00011a0 tx=0x7f3ec0007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.603+0000 7f3eca000640 1 -- 172.21.15.16:0/189736860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec000fa90 con 0x7f3ed415a2d0 2024-08-17T02:48:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.603+0000 7f3eca000640 1 -- 172.21.15.16:0/189736860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec000fbf0 con 0x7f3ed415a2d0 2024-08-17T02:48:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.603+0000 7f3eca000640 1 -- 172.21.15.16:0/189736860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0007050 con 0x7f3ed415a2d0 2024-08-17T02:48:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.603+0000 7f3ed9800640 1 -- 172.21.15.16:0/189736860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed415afe0 con 0x7f3ed415a2d0 2024-08-17T02:48:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.603+0000 7f3ed9800640 1 -- 172.21.15.16:0/189736860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed415b5a0 con 0x7f3ed415a2d0 2024-08-17T02:48:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.604+0000 7f3ed9800640 1 -- 172.21.15.16:0/189736860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed415baa0 con 0x7f3ed415a2d0 2024-08-17T02:48:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.605+0000 7f3eca000640 1 -- 172.21.15.16:0/189736860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f3ec00073c0 con 0x7f3ed415a2d0 2024-08-17T02:48:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.606+0000 7f3eca000640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3eb4065150 0x7f3eb4067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:35.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.606+0000 7f3eca000640 1 -- 172.21.15.16:0/189736860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f3ec0082c10 con 0x7f3ed415a2d0 2024-08-17T02:48:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.606+0000 7f3ed2a00640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3eb4065150 0x7f3eb4067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:35.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.607+0000 7f3ed2a00640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3eb4065150 0x7f3eb4067600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4034000 tx=0x7f3ec4035000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.610+0000 7f3eca000640 1 -- 172.21.15.16:0/189736860 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f3ec0050840 con 0x7f3ed415a2d0 2024-08-17T02:48:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.830+0000 7f3ed9800640 1 -- 172.21.15.16:0/189736860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ed4064db0 con 0x7f3ed415a2d0 2024-08-17T02:48:35.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.831+0000 7f3eca000640 1 -- 172.21.15.16:0/189736860 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5687 (secure 0 0 0) 0x7f3ec0053e60 con 0x7f3ed415a2d0 2024-08-17T02:48:35.832 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:35.832 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2889208700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2889208700},{"type":"v1","addr":"172.21.15.142:6837","nonce":2889208700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4287834697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4287834697},{"type":"v1","addr":"172.21.15.142:6835","nonce":4287834697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:16.190199+0000","modified":"2024-08-17T02:46:19.780153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4956},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4956":{"gid":4956,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.16:6835/219684854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":219684854},{"type":"v1","addr":"172.21.15.16:6835","nonce":219684854}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:28.938945+0000","modified":"2024-08-17T02:46:30.943869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6834/3115083427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3115083427},{"type":"v1","addr":"172.21.15.16:6834","nonce":3115083427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000521811","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:48:35.832 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 22 2024-08-17T02:48:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.833+0000 7f3ed9800640 1 -- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3eb4065150 msgr2=0x7f3eb4067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.833+0000 7f3ed9800640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3eb4065150 0x7f3eb4067600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4034000 tx=0x7f3ec4035000 comp rx=0 tx=0).stop 2024-08-17T02:48:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.833+0000 7f3ed9800640 1 -- 172.21.15.16:0/189736860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed415a2d0 msgr2=0x7f3ed415a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.833+0000 7f3ed9800640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed415a2d0 0x7f3ed415a720 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00011a0 tx=0x7f3ec0007ca0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.833+0000 7f3ed9800640 1 -- 172.21.15.16:0/189736860 shutdown_connections 2024-08-17T02:48:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.833+0000 7f3ed9800640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3eb4065150 0x7f3eb4067600 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.833+0000 7f3ed9800640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ed40a9920 0x7f3ed40a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.833+0000 7f3ed9800640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ed415a2d0 0x7f3ed415a720 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.833+0000 7f3ed9800640 1 --2- 172.21.15.16:0/189736860 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3eb8003b60 0x7f3ed41564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.833+0000 7f3ed9800640 1 -- 172.21.15.16:0/189736860 >> 172.21.15.16:0/189736860 conn(0x7f3ed40cb9b0 msgr2=0x7f3ed413f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:35.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.833+0000 7f3ed9800640 1 -- 172.21.15.16:0/189736860 shutdown_connections 2024-08-17T02:48:35.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:35.833+0000 7f3ed9800640 1 -- 172.21.15.16:0/189736860 wait complete. 2024-08-17T02:48:35.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:48:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.000+0000 7ff4dd000640 1 Processor -- start 2024-08-17T02:48:36.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.001+0000 7ff4dd000640 1 -- start start 2024-08-17T02:48:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.002+0000 7ff4dd000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d8069710 0x7ff4d80ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.002+0000 7ff4dd000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4d80ac540 0x7ff4d80af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.002+0000 7ff4dd000640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d80aec20 con 0x7ff4d8158bd0 2024-08-17T02:48:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.002+0000 7ff4dd000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d80ae330 con 0x7ff4d80ac540 2024-08-17T02:48:36.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.002+0000 7ff4dd000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d80acd30 con 0x7ff4d8069710 2024-08-17T02:48:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.002+0000 7ff4d7e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4d80ac540 0x7ff4d80af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.002+0000 7ff4d7e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4d80ac540 0x7ff4d80af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41764/0 (socket says 172.21.15.16:41764) 2024-08-17T02:48:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.002+0000 7ff4d7e00640 1 -- 172.21.15.16:0/117713329 learned_addr learned my addr 172.21.15.16:0/117713329 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.002+0000 7ff4d7400640 1 --1- 172.21.15.16:0/117713329 >> v1:172.21.15.16:6789/0 conn(0x7ff4d8158bd0 0x7ff4d8069000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45196/0 (socket says 172.21.15.16:45196) 2024-08-17T02:48:36.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.002+0000 7ff4cfe00640 1 --2- 172.21.15.16:0/117713329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d8069710 0x7ff4d80ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.003+0000 7ff4d7e00640 1 -- 172.21.15.16:0/117713329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d8069710 msgr2=0x7ff4d80ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.003+0000 7ff4d7e00640 1 --2- 172.21.15.16:0/117713329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d8069710 0x7ff4d80ac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.003+0000 7ff4d7e00640 1 -- 172.21.15.16:0/117713329 >> v1:172.21.15.16:6789/0 conn(0x7ff4d8158bd0 legacy=0x7ff4d8069000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.003+0000 7ff4d7e00640 1 -- 172.21.15.16:0/117713329 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d80ae5d0 con 0x7ff4d80ac540 2024-08-17T02:48:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.004+0000 7ff4d7e00640 1 --2- 172.21.15.16:0/117713329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d80ac540 0x7ff4d80af0b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009f20 tx=0x7ff4d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f406647fcff4e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.004+0000 7ff4d6a00640 1 -- 172.21.15.16:0/117713329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0031400 con 0x7ff4d80ac540 2024-08-17T02:48:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.004+0000 7ff4d6a00640 1 -- 172.21.15.16:0/117713329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d002d050 con 0x7ff4d80ac540 2024-08-17T02:48:36.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.004+0000 7ff4d6a00640 1 -- 172.21.15.16:0/117713329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00317d0 con 0x7ff4d80ac540 2024-08-17T02:48:36.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.004+0000 7ff4dd000640 1 -- 172.21.15.16:0/117713329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d80ac540 msgr2=0x7ff4d80af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.004+0000 7ff4dd000640 1 --2- 172.21.15.16:0/117713329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d80ac540 0x7ff4d80af0b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0009f20 tx=0x7ff4d0009840 comp rx=0 tx=0).stop 2024-08-17T02:48:36.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.004+0000 7ff4dd000640 1 -- 172.21.15.16:0/117713329 shutdown_connections 2024-08-17T02:48:36.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.004+0000 7ff4dd000640 1 --2- 172.21.15.16:0/117713329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d80ac540 0x7ff4d80af0b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.004+0000 7ff4dd000640 1 --2- 172.21.15.16:0/117713329 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d8069710 0x7ff4d80ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.005+0000 7ff4dd000640 1 -- 172.21.15.16:0/117713329 >> 172.21.15.16:0/117713329 conn(0x7ff4d8096a70 msgr2=0x7ff4d80cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:36.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.005+0000 7ff4dd000640 1 -- 172.21.15.16:0/117713329 shutdown_connections 2024-08-17T02:48:36.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.005+0000 7ff4dd000640 1 -- 172.21.15.16:0/117713329 wait complete. 2024-08-17T02:48:36.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.006+0000 7ff4dd000640 1 Processor -- start 2024-08-17T02:48:36.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.006+0000 7ff4dd000640 1 -- start start 2024-08-17T02:48:36.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.007+0000 7ff4dd000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8069710 0x7ff4d81532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.007+0000 7ff4dd000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d80ac540 0x7ff4d813f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.007+0000 7ff4dd000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d8158bd0 0x7ff4d813f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.007+0000 7ff4dd000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d813fd30 con 0x7ff4d80ac540 2024-08-17T02:48:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.007+0000 7ff4dd000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d8159480 con 0x7ff4d8069710 2024-08-17T02:48:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.007+0000 7ff4dd000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d815a000 con 0x7ff4d8158bd0 2024-08-17T02:48:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.007+0000 7ff4cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d80ac540 0x7ff4d813f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.007+0000 7ff4cfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d80ac540 0x7ff4d813f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41778/0 (socket says 172.21.15.16:41778) 2024-08-17T02:48:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.007+0000 7ff4cfe00640 1 -- 172.21.15.16:0/279365346 learned_addr learned my addr 172.21.15.16:0/279365346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.007+0000 7ff4d7400640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8069710 0x7ff4d81532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.007+0000 7ff4d7e00640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d8158bd0 0x7ff4d813f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.008+0000 7ff4cfe00640 1 -- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d8158bd0 msgr2=0x7ff4d813f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.008+0000 7ff4cfe00640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d8158bd0 0x7ff4d813f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.008+0000 7ff4cfe00640 1 -- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8069710 msgr2=0x7ff4d81532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.008+0000 7ff4cfe00640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8069710 0x7ff4d81532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.008+0000 7ff4cfe00640 1 -- 172.21.15.16:0/279365346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d0009ae0 con 0x7ff4d80ac540 2024-08-17T02:48:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.008+0000 7ff4d7400640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8069710 0x7ff4d81532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.008+0000 7ff4cfe00640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d80ac540 0x7ff4d813f2b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000d890 tx=0x7ff4c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.008+0000 7ff4cea00640 1 -- 172.21.15.16:0/279365346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c00096a0 con 0x7ff4d80ac540 2024-08-17T02:48:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.008+0000 7ff4cea00640 1 -- 172.21.15.16:0/279365346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c0009800 con 0x7ff4d80ac540 2024-08-17T02:48:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.008+0000 7ff4cea00640 1 -- 172.21.15.16:0/279365346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0009bc0 con 0x7ff4d80ac540 2024-08-17T02:48:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.008+0000 7ff4dd000640 1 -- 172.21.15.16:0/279365346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d81596d0 con 0x7ff4d80ac540 2024-08-17T02:48:36.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.008+0000 7ff4dd000640 1 -- 172.21.15.16:0/279365346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d8159c90 con 0x7ff4d80ac540 2024-08-17T02:48:36.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.010+0000 7ff4dd000640 1 -- 172.21.15.16:0/279365346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d8071050 con 0x7ff4d80ac540 2024-08-17T02:48:36.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.010+0000 7ff4cea00640 1 -- 172.21.15.16:0/279365346 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7ff4c0014620 con 0x7ff4d80ac540 2024-08-17T02:48:36.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.011+0000 7ff4cea00640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff4b4065150 0x7ff4b4067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.011+0000 7ff4cea00640 1 -- 172.21.15.16:0/279365346 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7ff4c00832c0 con 0x7ff4d80ac540 2024-08-17T02:48:36.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.011+0000 7ff4d7400640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff4b4065150 0x7ff4b4067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.012+0000 7ff4d7400640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff4b4065150 0x7ff4b4067600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80057b0 tx=0x7ff4c8007d80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:36.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.015+0000 7ff4cea00640 1 -- 172.21.15.16:0/279365346 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7ff4c0050ef0 con 0x7ff4d80ac540 2024-08-17T02:48:36.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.201+0000 7ff4dd000640 1 -- 172.21.15.16:0/279365346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4d815a140 con 0x7ff4d80ac540 2024-08-17T02:48:36.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.202+0000 7ff4cea00640 1 -- 172.21.15.16:0/279365346 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18788 (secure 0 0 0) 0x7ff4c0054510 con 0x7ff4d80ac540 2024-08-17T02:48:36.203 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:36.204 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:47:07.371280+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-08-17T02:46:14.322032+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-08-17T02:46:15.170652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":6,"pool_name":"cephfs.volume_0000000000521811.meta","create_time":"2024-08-17T02:46:27.102528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000521811"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000521811.data","create_time":"2024-08-17T02:46:27.975177+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000521811"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.e","mappings":[{"from":4,"to":5}]},{"pgid":"5.27","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]},{"pgid":"5.37","mappings":[{"from":4,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:48:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.204+0000 7ff4dd000640 1 -- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff4b4065150 msgr2=0x7ff4b4067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.204+0000 7ff4dd000640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff4b4065150 0x7ff4b4067600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80057b0 tx=0x7ff4c8007d80 comp rx=0 tx=0).stop 2024-08-17T02:48:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.204+0000 7ff4dd000640 1 -- 172.21.15.16:0/279365346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d80ac540 msgr2=0x7ff4d813f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.204+0000 7ff4dd000640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d80ac540 0x7ff4d813f2b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000d890 tx=0x7ff4c0008040 comp rx=0 tx=0).stop 2024-08-17T02:48:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.205+0000 7ff4dd000640 1 -- 172.21.15.16:0/279365346 shutdown_connections 2024-08-17T02:48:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.205+0000 7ff4dd000640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7ff4b4065150 0x7ff4b4067600 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.205+0000 7ff4dd000640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7ff4d8158bd0 0x7ff4d813f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.205+0000 7ff4dd000640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d80ac540 0x7ff4d813f2b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.205+0000 7ff4dd000640 1 --2- 172.21.15.16:0/279365346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4d8069710 0x7ff4d81532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.205+0000 7ff4dd000640 1 -- 172.21.15.16:0/279365346 >> 172.21.15.16:0/279365346 conn(0x7ff4d8096a70 msgr2=0x7ff4d80afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.205+0000 7ff4dd000640 1 -- 172.21.15.16:0/279365346 shutdown_connections 2024-08-17T02:48:36.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.205+0000 7ff4dd000640 1 -- 172.21.15.16:0/279365346 wait complete. 2024-08-17T02:48:36.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-08-17T02:48:36.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.373+0000 7f36e0200640 1 Processor -- start 2024-08-17T02:48:36.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.373+0000 7f36e0200640 1 -- start start 2024-08-17T02:48:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.374+0000 7f36e0200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36d8157250 0x7f36d8157620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.374+0000 7f36e0200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f36d815a300 0x7f36d80ac250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.374+0000 7f36e0200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d8158e60 con 0x7f36d8157b60 2024-08-17T02:48:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.374+0000 7f36e0200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d8158530 con 0x7f36d8157250 2024-08-17T02:48:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.374+0000 7f36e0200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d8158810 con 0x7f36d815a300 2024-08-17T02:48:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.374+0000 7f36dd800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36d8157250 0x7f36d8157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.374+0000 7f36dd800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36d8157250 0x7f36d8157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41792/0 (socket says 172.21.15.16:41792) 2024-08-17T02:48:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.374+0000 7f36dd800640 1 -- 172.21.15.16:0/948492255 learned_addr learned my addr 172.21.15.16:0/948492255 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.375+0000 7f36de200640 1 --2- 172.21.15.16:0/948492255 >> v2:172.21.15.142:3300/0 conn(0x7f36d815a300 0x7f36d80ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.375+0000 7f36dd800640 1 -- 172.21.15.16:0/948492255 >> v2:172.21.15.142:3300/0 conn(0x7f36d815a300 msgr2=0x7f36d80ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.375+0000 7f36dd800640 1 --2- 172.21.15.16:0/948492255 >> v2:172.21.15.142:3300/0 conn(0x7f36d815a300 0x7f36d80ac250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.375+0000 7f36dd800640 1 -- 172.21.15.16:0/948492255 >> v1:172.21.15.16:6789/0 conn(0x7f36d8157b60 legacy=0x7f36d80abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.375+0000 7f36dd800640 1 -- 172.21.15.16:0/948492255 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d8158af0 con 0x7f36d8157250 2024-08-17T02:48:36.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.375+0000 7f36d7e00640 1 -- 172.21.15.16:0/948492255 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573299637 0 0) 0x7f36d8158e60 con 0x7f36d8157b60 2024-08-17T02:48:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.376+0000 7f36dd800640 1 --2- 172.21.15.16:0/948492255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d8157250 0x7f36d8157620 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f36d80cdad0 tx=0x7f36cc0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a75182ef14f232 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.376+0000 7f36d7e00640 1 -- 172.21.15.16:0/948492255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc02ae80 con 0x7f36d8157250 2024-08-17T02:48:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.376+0000 7f36d7e00640 1 -- 172.21.15.16:0/948492255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36cc032440 con 0x7f36d8157250 2024-08-17T02:48:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.376+0000 7f36d7e00640 1 -- 172.21.15.16:0/948492255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0327c0 con 0x7f36d8157250 2024-08-17T02:48:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.377+0000 7f36e0200640 1 -- 172.21.15.16:0/948492255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d8157250 msgr2=0x7f36d8157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.377+0000 7f36e0200640 1 --2- 172.21.15.16:0/948492255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d8157250 0x7f36d8157620 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f36d80cdad0 tx=0x7f36cc0091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.377+0000 7f36e0200640 1 -- 172.21.15.16:0/948492255 shutdown_connections 2024-08-17T02:48:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.377+0000 7f36e0200640 1 --2- 172.21.15.16:0/948492255 >> v2:172.21.15.142:3300/0 conn(0x7f36d815a300 0x7f36d80ac250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.377+0000 7f36e0200640 1 --2- 172.21.15.16:0/948492255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d8157250 0x7f36d8157620 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.377+0000 7f36e0200640 1 -- 172.21.15.16:0/948492255 >> 172.21.15.16:0/948492255 conn(0x7f36d8096a70 msgr2=0x7f36d8096e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.377+0000 7f36e0200640 1 -- 172.21.15.16:0/948492255 shutdown_connections 2024-08-17T02:48:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.377+0000 7f36e0200640 1 -- 172.21.15.16:0/948492255 wait complete. 2024-08-17T02:48:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.378+0000 7f36e0200640 1 Processor -- start 2024-08-17T02:48:36.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.379+0000 7f36e0200640 1 -- start start 2024-08-17T02:48:36.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.379+0000 7f36e0200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36d8157250 0x7f36d8150990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.379+0000 7f36e0200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d8157b60 0x7f36d8156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.379+0000 7f36e0200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36d815a300 0x7f36d80ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.379+0000 7f36e0200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d8169550 con 0x7f36d8157b60 2024-08-17T02:48:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.379+0000 7f36e0200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d816a360 con 0x7f36d815a300 2024-08-17T02:48:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.379+0000 7f36e0200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d8169790 con 0x7f36d8157250 2024-08-17T02:48:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36dce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d8157b60 0x7f36d8156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36dce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d8157b60 0x7f36d8156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41808/0 (socket says 172.21.15.16:41808) 2024-08-17T02:48:36.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36dce00640 1 -- 172.21.15.16:0/2444525137 learned_addr learned my addr 172.21.15.16:0/2444525137 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:36.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36de200640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36d815a300 0x7f36d80ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36dce00640 1 -- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36d8157250 msgr2=0x7f36d8150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36dd800640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36d8157250 0x7f36d8150990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36dce00640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36d8157250 0x7f36d8150990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36dce00640 1 -- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36d815a300 msgr2=0x7f36d80ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36dce00640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36d815a300 0x7f36d80ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36dce00640 1 -- 172.21.15.16:0/2444525137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36cc009450 con 0x7f36d8157b60 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36de200640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36d815a300 0x7f36d80ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36dd800640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36d8157250 0x7f36d8150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.380+0000 7f36dce00640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d8157b60 0x7f36d8156200 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f36c400c5b0 tx=0x7f36c4007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.381+0000 7f36d4c00640 1 -- 172.21.15.16:0/2444525137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4012630 con 0x7f36d8157b60 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.381+0000 7f36d4c00640 1 -- 172.21.15.16:0/2444525137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c40127d0 con 0x7f36d8157b60 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.381+0000 7f36d4c00640 1 -- 172.21.15.16:0/2444525137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4012b90 con 0x7f36d8157b60 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.381+0000 7f36e0200640 1 -- 172.21.15.16:0/2444525137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d8169a30 con 0x7f36d8157b60 2024-08-17T02:48:36.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.381+0000 7f36e0200640 1 -- 172.21.15.16:0/2444525137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d8169ff0 con 0x7f36d8157b60 2024-08-17T02:48:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.382+0000 7f36e0200640 1 -- 172.21.15.16:0/2444525137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d816a4a0 con 0x7f36d8157b60 2024-08-17T02:48:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.382+0000 7f36d4c00640 1 -- 172.21.15.16:0/2444525137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f36c401b400 con 0x7f36d8157b60 2024-08-17T02:48:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.383+0000 7f36d4c00640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f36b8065180 0x7f36b8067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.383+0000 7f36d4c00640 1 -- 172.21.15.16:0/2444525137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6719+0+0 (secure 0 0 0) 0x7f36c4082450 con 0x7f36d8157b60 2024-08-17T02:48:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.384+0000 7f36dd800640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f36b8065180 0x7f36b8067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.384+0000 7f36dd800640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f36b8065180 0x7f36b8067630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f36cc034000 tx=0x7f36cc032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:36.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.387+0000 7f36d4c00640 1 -- 172.21.15.16:0/2444525137 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f36d816a4a0 con 0x7f36d8157b60 2024-08-17T02:48:36.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.606+0000 7f36e0200640 1 -- 172.21.15.16:0/2444525137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f36d816a4a0 con 0x7f36d8157b60 2024-08-17T02:48:36.607 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-08-17T02:48:36.606+0000 7f23cec00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-17T02:48:36.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.611+0000 7f36d4c00640 1 -- 172.21.15.16:0/2444525137 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7f36d816a4a0 con 0x7f36d8157b60 2024-08-17T02:48:36.612 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-17T02:48:36.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.614+0000 7f36e0200640 1 -- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f36b8065180 msgr2=0x7f36b8067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.614+0000 7f36e0200640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f36b8065180 0x7f36b8067630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f36cc034000 tx=0x7f36cc032000 comp rx=0 tx=0).stop 2024-08-17T02:48:36.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.614+0000 7f36e0200640 1 -- 172.21.15.16:0/2444525137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d8157b60 msgr2=0x7f36d8156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.614+0000 7f36e0200640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d8157b60 0x7f36d8156200 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f36c400c5b0 tx=0x7f36c4007d60 comp rx=0 tx=0).stop 2024-08-17T02:48:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.614+0000 7f36e0200640 1 -- 172.21.15.16:0/2444525137 shutdown_connections 2024-08-17T02:48:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.614+0000 7f36e0200640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f36b8065180 0x7f36b8067630 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.614+0000 7f36e0200640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f36d815a300 0x7f36d80ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.614+0000 7f36e0200640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36d8157b60 0x7f36d8156200 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.614+0000 7f36e0200640 1 --2- 172.21.15.16:0/2444525137 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f36d8157250 0x7f36d8150990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.614+0000 7f36e0200640 1 -- 172.21.15.16:0/2444525137 >> 172.21.15.16:0/2444525137 conn(0x7f36d8096a70 msgr2=0x7f36d80b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.614+0000 7f36e0200640 1 -- 172.21.15.16:0/2444525137 shutdown_connections 2024-08-17T02:48:36.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.614+0000 7f36e0200640 1 -- 172.21.15.16:0/2444525137 wait complete. 2024-08-17T02:48:36.628 INFO:tasks.ceph.mds.a.smithi016.stderr: -7281> 2024-08-17T02:46:18.721+0000 7f06c6400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-17T02:48:36.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-08-17T02:48:36.676 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2024-08-17T02:48:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.795+0000 7f060b200640 1 Processor -- start 2024-08-17T02:48:36.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.795+0000 7f060b200640 1 -- start start 2024-08-17T02:48:36.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.796+0000 7f060b200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f06041591b0 0x7f06040cd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.796+0000 7f060b200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06040cd950 con 0x7f0604158de0 2024-08-17T02:48:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.796+0000 7f060b200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06040ccf80 con 0x7f0604068600 2024-08-17T02:48:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.796+0000 7f060b200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06040ac3d0 con 0x7f06041591b0 2024-08-17T02:48:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.797+0000 7f0603e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0604158de0 0x7f06040cb9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45214/0 (socket says 172.21.15.16:45214) 2024-08-17T02:48:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.797+0000 7f0603e00640 1 -- 172.21.15.16:0/3864365420 learned_addr learned my addr 172.21.15.16:0/3864365420 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:36.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.797+0000 7f05fbe00640 1 --2- 172.21.15.16:0/3864365420 >> v2:172.21.15.142:3300/0 conn(0x7f06041591b0 0x7f06040cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.797+0000 7f0603400640 1 -- 172.21.15.16:0/3864365420 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335593545 0 0) 0x7f05f4001110 con 0x7f0604158de0 2024-08-17T02:48:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.797+0000 7f0603400640 1 -- 172.21.15.16:0/3864365420 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e80034e0 con 0x7f0604158de0 2024-08-17T02:48:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.797+0000 7f05fbe00640 1 -- 172.21.15.16:0/3864365420 >> v1:172.21.15.142:6789/0 conn(0x7f0604068600 legacy=0x7f0604068a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.797+0000 7f05fbe00640 1 -- 172.21.15.16:0/3864365420 >> v1:172.21.15.16:6789/0 conn(0x7f0604158de0 legacy=0x7f06040cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.797+0000 7f05fbe00640 1 -- 172.21.15.16:0/3864365420 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040cc100 con 0x7f06041591b0 2024-08-17T02:48:36.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.798+0000 7f05fbe00640 1 --2- 172.21.15.16:0/3864365420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06041591b0 0x7f06040cd410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009850 tx=0x7f05ec0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d38583cd770dab5c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.799+0000 7f0603400640 1 -- 172.21.15.16:0/3864365420 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec02ae20 con 0x7f06041591b0 2024-08-17T02:48:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.799+0000 7f0603400640 1 -- 172.21.15.16:0/3864365420 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec032440 con 0x7f06041591b0 2024-08-17T02:48:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.799+0000 7f0603400640 1 -- 172.21.15.16:0/3864365420 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec0327e0 con 0x7f06041591b0 2024-08-17T02:48:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.799+0000 7f060b200640 1 -- 172.21.15.16:0/3864365420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06041591b0 msgr2=0x7f06040cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.799+0000 7f060b200640 1 --2- 172.21.15.16:0/3864365420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06041591b0 0x7f06040cd410 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009850 tx=0x7f05ec0090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.799+0000 7f060b200640 1 -- 172.21.15.16:0/3864365420 shutdown_connections 2024-08-17T02:48:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.799+0000 7f060b200640 1 --2- 172.21.15.16:0/3864365420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06041591b0 0x7f06040cd410 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.799+0000 7f060b200640 1 -- 172.21.15.16:0/3864365420 >> 172.21.15.16:0/3864365420 conn(0x7f0604096a70 msgr2=0x7f0604096e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.799+0000 7f060b200640 1 -- 172.21.15.16:0/3864365420 shutdown_connections 2024-08-17T02:48:36.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.799+0000 7f060b200640 1 -- 172.21.15.16:0/3864365420 wait complete. 2024-08-17T02:48:36.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.801+0000 7f060b200640 1 Processor -- start 2024-08-17T02:48:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.801+0000 7f060b200640 1 -- start start 2024-08-17T02:48:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f060b200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0604068600 0x7f06040ac510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f060b200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0604158de0 0x7f060414b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f060b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06041591b0 0x7f06040af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f060b200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060414bb70 con 0x7f0604158de0 2024-08-17T02:48:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f060b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06041697a0 con 0x7f06041591b0 2024-08-17T02:48:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f060b200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06040af820 con 0x7f0604068600 2024-08-17T02:48:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f05fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0604158de0 0x7f060414b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f05fbe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0604158de0 0x7f060414b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41828/0 (socket says 172.21.15.16:41828) 2024-08-17T02:48:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f05fbe00640 1 -- 172.21.15.16:0/2654340493 learned_addr learned my addr 172.21.15.16:0/2654340493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f0603e00640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0604068600 0x7f06040ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f0609200640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06041591b0 0x7f06040af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f05fbe00640 1 -- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0604068600 msgr2=0x7f06040ac510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f05fbe00640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0604068600 0x7f06040ac510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f05fbe00640 1 -- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06041591b0 msgr2=0x7f06040af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f05fbe00640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06041591b0 0x7f06040af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f05fbe00640 1 -- 172.21.15.16:0/2654340493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec009390 con 0x7f0604158de0 2024-08-17T02:48:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.802+0000 7f0609200640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06041591b0 0x7f06040af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:36.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.803+0000 7f05fbe00640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0604158de0 0x7f060414b630 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009360 tx=0x7f05ec032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:36.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.803+0000 7f05fb400640 1 -- 172.21.15.16:0/2654340493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec000d10 con 0x7f0604158de0 2024-08-17T02:48:36.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.803+0000 7f05fb400640 1 -- 172.21.15.16:0/2654340493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec031e00 con 0x7f0604158de0 2024-08-17T02:48:36.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.803+0000 7f05fb400640 1 -- 172.21.15.16:0/2654340493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec03d670 con 0x7f0604158de0 2024-08-17T02:48:36.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.803+0000 7f060b200640 1 -- 172.21.15.16:0/2654340493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06040acaf0 con 0x7f0604158de0 2024-08-17T02:48:36.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.803+0000 7f060b200640 1 -- 172.21.15.16:0/2654340493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0604169d20 con 0x7f0604158de0 2024-08-17T02:48:36.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.804+0000 7f060b200640 1 -- 172.21.15.16:0/2654340493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0604071050 con 0x7f0604158de0 2024-08-17T02:48:36.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.805+0000 7f05fb400640 1 -- 172.21.15.16:0/2654340493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f05ec039040 con 0x7f0604158de0 2024-08-17T02:48:36.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.806+0000 7f05fb400640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f05e0065150 0x7f05e0067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:36.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.806+0000 7f05fb400640 1 -- 172.21.15.16:0/2654340493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6805+0+0 (secure 0 0 0) 0x7f05ec0a28e0 con 0x7f0604158de0 2024-08-17T02:48:36.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.806+0000 7f0603e00640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f05e0065150 0x7f05e0067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:36.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.807+0000 7f0603e00640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f05e0065150 0x7f05e0067600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f05f4001270 tx=0x7f05f4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:36.810+0000 7f05fb400640 1 -- 172.21.15.16:0/2654340493 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f05ec0704b0 con 0x7f0604158de0 2024-08-17T02:48:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.034+0000 7f060b200640 1 -- 172.21.15.16:0/2654340493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f06040704a0 con 0x7f0604158de0 2024-08-17T02:48:37.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.611+0000 7f05fb400640 1 -- 172.21.15.16:0/2654340493 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7f05ec073ad0 con 0x7f0604158de0 2024-08-17T02:48:37.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.614+0000 7f060b200640 1 -- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f05e0065150 msgr2=0x7f05e0067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.614+0000 7f060b200640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f05e0065150 0x7f05e0067600 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f05f4001270 tx=0x7f05f4009040 comp rx=0 tx=0).stop 2024-08-17T02:48:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.614+0000 7f060b200640 1 -- 172.21.15.16:0/2654340493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0604158de0 msgr2=0x7f060414b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.614+0000 7f060b200640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0604158de0 0x7f060414b630 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009360 tx=0x7f05ec032000 comp rx=0 tx=0).stop 2024-08-17T02:48:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.614+0000 7f060b200640 1 -- 172.21.15.16:0/2654340493 shutdown_connections 2024-08-17T02:48:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.614+0000 7f060b200640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f05e0065150 0x7f05e0067600 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.614+0000 7f060b200640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06041591b0 0x7f06040af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.614+0000 7f060b200640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0604158de0 0x7f060414b630 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.614+0000 7f060b200640 1 --2- 172.21.15.16:0/2654340493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0604068600 0x7f06040ac510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.614+0000 7f060b200640 1 -- 172.21.15.16:0/2654340493 >> 172.21.15.16:0/2654340493 conn(0x7f0604096a70 msgr2=0x7f060406fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.615+0000 7f060b200640 1 -- 172.21.15.16:0/2654340493 shutdown_connections 2024-08-17T02:48:37.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.615+0000 7f060b200640 1 -- 172.21.15.16:0/2654340493 wait complete. 2024-08-17T02:48:37.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-17T02:48:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.833+0000 7f38f7200640 1 Processor -- start 2024-08-17T02:48:37.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.834+0000 7f38f7200640 1 -- start start 2024-08-17T02:48:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.835+0000 7f38f7200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38f8158f30 0x7f38f80a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.835+0000 7f38f7200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f80aa960 con 0x7f38f80a9e20 2024-08-17T02:48:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.835+0000 7f38f7200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f80ab6a0 con 0x7f38f8158b60 2024-08-17T02:48:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.835+0000 7f38f7200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f80ac240 con 0x7f38f8158f30 2024-08-17T02:48:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.835+0000 7f38f5400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38f8158f30 0x7f38f80a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:37.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.835+0000 7f38f5400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38f8158f30 0x7f38f80a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41844/0 (socket says 172.21.15.16:41844) 2024-08-17T02:48:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.835+0000 7f38f5400640 1 -- 172.21.15.16:0/1549819621 learned_addr learned my addr 172.21.15.16:0/1549819621 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.836+0000 7f38f5400640 1 -- 172.21.15.16:0/1549819621 >> v1:172.21.15.142:6789/0 conn(0x7f38f8158b60 legacy=0x7f38f80abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.836+0000 7f38f5400640 1 -- 172.21.15.16:0/1549819621 >> v1:172.21.15.16:6789/0 conn(0x7f38f80a9e20 legacy=0x7f38f80ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:37.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.836+0000 7f38f5400640 1 -- 172.21.15.16:0/1549819621 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f80aa3a0 con 0x7f38f8158f30 2024-08-17T02:48:37.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.837+0000 7f38f5400640 1 --2- 172.21.15.16:0/1549819621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8158f30 0x7f38f80a98e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009910 tx=0x7f38e40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30482ebee4c9f509 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:37.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.837+0000 7f38f4a00640 1 -- 172.21.15.16:0/1549819621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4032440 con 0x7f38f8158f30 2024-08-17T02:48:37.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.837+0000 7f38f4a00640 1 -- 172.21.15.16:0/1549819621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e40325a0 con 0x7f38f8158f30 2024-08-17T02:48:37.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.837+0000 7f38f4a00640 1 -- 172.21.15.16:0/1549819621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4032980 con 0x7f38f8158f30 2024-08-17T02:48:37.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.837+0000 7f38f7200640 1 -- 172.21.15.16:0/1549819621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8158f30 msgr2=0x7f38f80a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:37.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.837+0000 7f38f7200640 1 --2- 172.21.15.16:0/1549819621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8158f30 0x7f38f80a98e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009910 tx=0x7f38e40091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:37.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.837+0000 7f38f7200640 1 -- 172.21.15.16:0/1549819621 shutdown_connections 2024-08-17T02:48:37.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.837+0000 7f38f7200640 1 --2- 172.21.15.16:0/1549819621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f8158f30 0x7f38f80a98e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:37.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.837+0000 7f38f7200640 1 -- 172.21.15.16:0/1549819621 >> 172.21.15.16:0/1549819621 conn(0x7f38f8067750 msgr2=0x7f38f80cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:37.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.838+0000 7f38f7200640 1 -- 172.21.15.16:0/1549819621 shutdown_connections 2024-08-17T02:48:37.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.838+0000 7f38f7200640 1 -- 172.21.15.16:0/1549819621 wait complete. 2024-08-17T02:48:37.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.839+0000 7f38f7200640 1 Processor -- start 2024-08-17T02:48:37.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.839+0000 7f38f7200640 1 -- start start 2024-08-17T02:48:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.840+0000 7f38f7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f80a9e20 0x7f38f81517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.840+0000 7f38f7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f8158b60 0x7f38f815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.840+0000 7f38f7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38f8158f30 0x7f38f815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.840+0000 7f38f7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f815a300 con 0x7f38f80a9e20 2024-08-17T02:48:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.840+0000 7f38f7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f8159410 con 0x7f38f8158b60 2024-08-17T02:48:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.840+0000 7f38f7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f81596f0 con 0x7f38f8158f30 2024-08-17T02:48:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.840+0000 7f38f5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f80a9e20 0x7f38f81517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.840+0000 7f38f5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f80a9e20 0x7f38f81517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41858/0 (socket says 172.21.15.16:41858) 2024-08-17T02:48:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.840+0000 7f38f5e00640 1 -- 172.21.15.16:0/1103454552 learned_addr learned my addr 172.21.15.16:0/1103454552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.840+0000 7f38f5400640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f8158b60 0x7f38f815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:37.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.840+0000 7f38f6800640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38f8158f30 0x7f38f815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.841+0000 7f38f5e00640 1 -- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38f8158f30 msgr2=0x7f38f815b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.841+0000 7f38f5e00640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38f8158f30 0x7f38f815b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.841+0000 7f38f5e00640 1 -- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f8158b60 msgr2=0x7f38f815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.841+0000 7f38f5e00640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f8158b60 0x7f38f815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.841+0000 7f38f5e00640 1 -- 172.21.15.16:0/1103454552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4009450 con 0x7f38f80a9e20 2024-08-17T02:48:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.841+0000 7f38f6800640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38f8158f30 0x7f38f815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:37.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.841+0000 7f38f5e00640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f80a9e20 0x7f38f81517b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c560 tx=0x7f38e8007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.841+0000 7f38ed600640 1 -- 172.21.15.16:0/1103454552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8011590 con 0x7f38f80a9e20 2024-08-17T02:48:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.841+0000 7f38ed600640 1 -- 172.21.15.16:0/1103454552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e8011710 con 0x7f38f80a9e20 2024-08-17T02:48:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.841+0000 7f38ed600640 1 -- 172.21.15.16:0/1103454552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8011ad0 con 0x7f38f80a9e20 2024-08-17T02:48:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.841+0000 7f38f7200640 1 -- 172.21.15.16:0/1103454552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f81599d0 con 0x7f38f80a9e20 2024-08-17T02:48:37.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.841+0000 7f38f7200640 1 -- 172.21.15.16:0/1103454552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f8159f90 con 0x7f38f80a9e20 2024-08-17T02:48:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.843+0000 7f38f7200640 1 -- 172.21.15.16:0/1103454552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f8071050 con 0x7f38f80a9e20 2024-08-17T02:48:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.843+0000 7f38ed600640 1 -- 172.21.15.16:0/1103454552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f38e801b3b0 con 0x7f38f80a9e20 2024-08-17T02:48:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.844+0000 7f38ed600640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f38d8065150 0x7f38d8067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.844+0000 7f38ed600640 1 -- 172.21.15.16:0/1103454552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6805+0+0 (secure 0 0 0) 0x7f38e8081e30 con 0x7f38f80a9e20 2024-08-17T02:48:37.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.845+0000 7f38f5400640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f38d8065150 0x7f38d8067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:37.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.845+0000 7f38f5400640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f38d8065150 0x7f38d8067600 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f38e4034000 tx=0x7f38e4032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:37.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:37.848+0000 7f38ed600640 1 -- 172.21.15.16:0/1103454552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f38e804fab0 con 0x7f38f80a9e20 2024-08-17T02:48:38.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.031+0000 7f38f7200640 1 -- 172.21.15.16:0/1103454552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38f80ae200 con 0x7f38f80a9e20 2024-08-17T02:48:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.616+0000 7f38ed600640 1 -- 172.21.15.16:0/1103454552 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) v1 ==== 181+0+0 (secure 0 0 0) 0x7f38e8017070 con 0x7f38f80a9e20 2024-08-17T02:48:38.617 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-08-17T02:48:38.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.619+0000 7f38f7200640 1 -- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f38d8065150 msgr2=0x7f38d8067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:38.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.619+0000 7f38f7200640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f38d8065150 0x7f38d8067600 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f38e4034000 tx=0x7f38e4032000 comp rx=0 tx=0).stop 2024-08-17T02:48:38.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.619+0000 7f38f7200640 1 -- 172.21.15.16:0/1103454552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f80a9e20 msgr2=0x7f38f81517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:38.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.619+0000 7f38f7200640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f80a9e20 0x7f38f81517b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c560 tx=0x7f38e8007ef0 comp rx=0 tx=0).stop 2024-08-17T02:48:38.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.619+0000 7f38f7200640 1 -- 172.21.15.16:0/1103454552 shutdown_connections 2024-08-17T02:48:38.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.619+0000 7f38f7200640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f38d8065150 0x7f38d8067600 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:38.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.619+0000 7f38f7200640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f38f8158f30 0x7f38f815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:38.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.619+0000 7f38f7200640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f8158b60 0x7f38f815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:38.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.619+0000 7f38f7200640 1 --2- 172.21.15.16:0/1103454552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38f80a9e20 0x7f38f81517b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:38.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.619+0000 7f38f7200640 1 -- 172.21.15.16:0/1103454552 >> 172.21.15.16:0/1103454552 conn(0x7f38f8067750 msgr2=0x7f38f81564b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:38.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.619+0000 7f38f7200640 1 -- 172.21.15.16:0/1103454552 shutdown_connections 2024-08-17T02:48:38.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.619+0000 7f38f7200640 1 -- 172.21.15.16:0/1103454552 wait complete. 2024-08-17T02:48:38.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-17T02:48:38.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.794+0000 7f94d1c00640 1 Processor -- start 2024-08-17T02:48:38.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.794+0000 7f94d1c00640 1 -- start start 2024-08-17T02:48:38.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.795+0000 7f94d1c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94cc156d40 0x7f94cc06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:38.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.795+0000 7f94d1c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f94cc06e420 0x7f94cc0ab510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.795+0000 7f94d1c00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94cc1568b0 con 0x7f94cc150130 2024-08-17T02:48:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.795+0000 7f94d1c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94cc06e8a0 con 0x7f94cc06e420 2024-08-17T02:48:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.795+0000 7f94d1c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc150500 con 0x7f94cc156d40 2024-08-17T02:48:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.796+0000 7f94caa00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f94cc150130 0x7f94cc0aa730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45240/0 (socket says 172.21.15.16:45240) 2024-08-17T02:48:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.796+0000 7f94caa00640 1 -- 172.21.15.16:0/1337026397 learned_addr learned my addr 172.21.15.16:0/1337026397 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.796+0000 7f94cb400640 1 --2- 172.21.15.16:0/1337026397 >> v2:172.21.15.142:3300/0 conn(0x7f94cc06e420 0x7f94cc0ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.796+0000 7f94ca000640 1 --2- 172.21.15.16:0/1337026397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94cc156d40 0x7f94cc06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.796+0000 7f94c9600640 1 -- 172.21.15.16:0/1337026397 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429534489 0 0) 0x7f94cc1568b0 con 0x7f94cc150130 2024-08-17T02:48:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.796+0000 7f94c9600640 1 -- 172.21.15.16:0/1337026397 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94b00034e0 con 0x7f94cc150130 2024-08-17T02:48:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.796+0000 7f94c9600640 1 -- 172.21.15.16:0/1337026397 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3533768752 0 0) 0x7f94b00034e0 con 0x7f94cc150130 2024-08-17T02:48:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.796+0000 7f94cb400640 1 -- 172.21.15.16:0/1337026397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94cc156d40 msgr2=0x7f94cc06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:38.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.796+0000 7f94cb400640 1 --2- 172.21.15.16:0/1337026397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94cc156d40 0x7f94cc06dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:38.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.796+0000 7f94cb400640 1 -- 172.21.15.16:0/1337026397 >> v1:172.21.15.16:6789/0 conn(0x7f94cc150130 legacy=0x7f94cc0aa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:38.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.796+0000 7f94cb400640 1 -- 172.21.15.16:0/1337026397 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc155fc0 con 0x7f94cc06e420 2024-08-17T02:48:38.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.796+0000 7f94ca000640 1 --2- 172.21.15.16:0/1337026397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94cc156d40 0x7f94cc06dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:48:38.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.797+0000 7f94cb400640 1 --2- 172.21.15.16:0/1337026397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94cc06e420 0x7f94cc0ab510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009e60 tx=0x7f94c4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b05065281b03779b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:38.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.798+0000 7f94c9600640 1 -- 172.21.15.16:0/1337026397 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc001470 con 0x7f94cc06e420 2024-08-17T02:48:38.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.798+0000 7f94c9600640 1 -- 172.21.15.16:0/1337026397 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c402d050 con 0x7f94cc06e420 2024-08-17T02:48:38.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.798+0000 7f94c9600640 1 -- 172.21.15.16:0/1337026397 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4031610 con 0x7f94cc06e420 2024-08-17T02:48:38.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.798+0000 7f94d1c00640 1 -- 172.21.15.16:0/1337026397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94cc06e420 msgr2=0x7f94cc0ab510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:38.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.798+0000 7f94d1c00640 1 --2- 172.21.15.16:0/1337026397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94cc06e420 0x7f94cc0ab510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009e60 tx=0x7f94c4009780 comp rx=0 tx=0).stop 2024-08-17T02:48:38.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.798+0000 7f94d1c00640 1 -- 172.21.15.16:0/1337026397 shutdown_connections 2024-08-17T02:48:38.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.798+0000 7f94d1c00640 1 --2- 172.21.15.16:0/1337026397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94cc06e420 0x7f94cc0ab510 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:38.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.798+0000 7f94d1c00640 1 --2- 172.21.15.16:0/1337026397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94cc156d40 0x7f94cc06dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:38.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.798+0000 7f94d1c00640 1 -- 172.21.15.16:0/1337026397 >> 172.21.15.16:0/1337026397 conn(0x7f94cc067750 msgr2=0x7f94cc06c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:38.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.799+0000 7f94d1c00640 1 -- 172.21.15.16:0/1337026397 shutdown_connections 2024-08-17T02:48:38.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.799+0000 7f94d1c00640 1 -- 172.21.15.16:0/1337026397 wait complete. 2024-08-17T02:48:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.800+0000 7f94d1c00640 1 Processor -- start 2024-08-17T02:48:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.800+0000 7f94d1c00640 1 -- start start 2024-08-17T02:48:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.801+0000 7f94d1c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94cc06e420 0x7f94cc13def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.801+0000 7f94d1c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94cc150130 0x7f94cc0a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.801+0000 7f94d1c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94cc156d40 0x7f94cc0a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.801+0000 7f94d1c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc169bc0 con 0x7f94cc06e420 2024-08-17T02:48:38.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.801+0000 7f94d1c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc0a6330 con 0x7f94cc156d40 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.801+0000 7f94d1c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc169e10 con 0x7f94cc150130 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.801+0000 7f94caa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94cc06e420 0x7f94cc13def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.801+0000 7f94caa00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94cc06e420 0x7f94cc13def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41868/0 (socket says 172.21.15.16:41868) 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.801+0000 7f94caa00640 1 -- 172.21.15.16:0/2002305571 learned_addr learned my addr 172.21.15.16:0/2002305571 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.801+0000 7f94ca000640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94cc150130 0x7f94cc0a5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.802+0000 7f94cb400640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94cc156d40 0x7f94cc0a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.802+0000 7f94caa00640 1 -- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94cc150130 msgr2=0x7f94cc0a5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.802+0000 7f94caa00640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94cc150130 0x7f94cc0a5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.802+0000 7f94caa00640 1 -- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94cc156d40 msgr2=0x7f94cc0a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.802+0000 7f94caa00640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94cc156d40 0x7f94cc0a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.802+0000 7f94caa00640 1 -- 172.21.15.16:0/2002305571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8009390 con 0x7f94cc06e420 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.802+0000 7f94cb400640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94cc156d40 0x7f94cc0a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.802+0000 7f94caa00640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94cc06e420 0x7f94cc13def0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00cb90 tx=0x7f94bc001b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:38.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.802+0000 7f94c2000640 1 -- 172.21.15.16:0/2002305571 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc007360 con 0x7f94cc06e420 2024-08-17T02:48:38.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.802+0000 7f94c2000640 1 -- 172.21.15.16:0/2002305571 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94bc0074e0 con 0x7f94cc06e420 2024-08-17T02:48:38.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.802+0000 7f94c2000640 1 -- 172.21.15.16:0/2002305571 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0049f0 con 0x7f94cc06e420 2024-08-17T02:48:38.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.802+0000 7f94d1c00640 1 -- 172.21.15.16:0/2002305571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c4009a20 con 0x7f94cc06e420 2024-08-17T02:48:38.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.803+0000 7f94d1c00640 1 -- 172.21.15.16:0/2002305571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94cc0c9530 con 0x7f94cc06e420 2024-08-17T02:48:38.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.804+0000 7f94d1c00640 1 -- 172.21.15.16:0/2002305571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94cc0710c0 con 0x7f94cc06e420 2024-08-17T02:48:38.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.804+0000 7f94c2000640 1 -- 172.21.15.16:0/2002305571 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f94bc00a610 con 0x7f94cc06e420 2024-08-17T02:48:38.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.805+0000 7f94c2000640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f94ac065150 0x7f94ac067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:38.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.805+0000 7f94c2000640 1 -- 172.21.15.16:0/2002305571 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6367+0+0 (secure 0 0 0) 0x7f94bc081b10 con 0x7f94cc06e420 2024-08-17T02:48:38.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.806+0000 7f94ca000640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f94ac065150 0x7f94ac067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:38.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.807+0000 7f94ca000640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f94ac065150 0x7f94ac067600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f94cc155f00 tx=0x7f94b80090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:38.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:38.809+0000 7f94c2000640 1 -- 172.21.15.16:0/2002305571 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f94bc04f9d0 con 0x7f94cc06e420 2024-08-17T02:48:39.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.000+0000 7f94d1c00640 1 -- 172.21.15.16:0/2002305571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94cc0c8570 con 0x7f94cc06e420 2024-08-17T02:48:39.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.622+0000 7f94c2000640 1 -- 172.21.15.16:0/2002305571 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) v1 ==== 169+0+0 (secure 0 0 0) 0x7f94bc018020 con 0x7f94cc06e420 2024-08-17T02:48:39.624 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-08-17T02:48:39.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.628+0000 7f94d1c00640 1 -- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f94ac065150 msgr2=0x7f94ac067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:39.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.628+0000 7f94d1c00640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f94ac065150 0x7f94ac067600 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f94cc155f00 tx=0x7f94b80090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:39.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.628+0000 7f94d1c00640 1 -- 172.21.15.16:0/2002305571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94cc06e420 msgr2=0x7f94cc13def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:39.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.628+0000 7f94d1c00640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94cc06e420 0x7f94cc13def0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00cb90 tx=0x7f94bc001b30 comp rx=0 tx=0).stop 2024-08-17T02:48:39.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.628+0000 7f94d1c00640 1 -- 172.21.15.16:0/2002305571 shutdown_connections 2024-08-17T02:48:39.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.629+0000 7f94d1c00640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f94ac065150 0x7f94ac067600 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:39.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.629+0000 7f94d1c00640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f94cc156d40 0x7f94cc0a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:39.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.629+0000 7f94d1c00640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f94cc150130 0x7f94cc0a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:39.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.629+0000 7f94d1c00640 1 --2- 172.21.15.16:0/2002305571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94cc06e420 0x7f94cc13def0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:39.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.629+0000 7f94d1c00640 1 -- 172.21.15.16:0/2002305571 >> 172.21.15.16:0/2002305571 conn(0x7f94cc067750 msgr2=0x7f94cc06d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:39.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.629+0000 7f94d1c00640 1 -- 172.21.15.16:0/2002305571 shutdown_connections 2024-08-17T02:48:39.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.629+0000 7f94d1c00640 1 -- 172.21.15.16:0/2002305571 wait complete. 2024-08-17T02:48:39.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:48:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.807+0000 7f7d8ea00640 1 Processor -- start 2024-08-17T02:48:39.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.807+0000 7f7d8ea00640 1 -- start start 2024-08-17T02:48:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.808+0000 7f7d8ea00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7d880b1440 0x7f7d880caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:39.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.808+0000 7f7d8ea00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d88140580 0x7f7d880c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.808+0000 7f7d8ea00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d880caa80 con 0x7f7d881401b0 2024-08-17T02:48:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.808+0000 7f7d8ea00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d880cb450 con 0x7f7d88140580 2024-08-17T02:48:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.808+0000 7f7d8ea00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d880c9e70 con 0x7f7d880b1440 2024-08-17T02:48:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.809+0000 7f7d8ca00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d88140580 0x7f7d880c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.809+0000 7f7d8ca00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d88140580 0x7f7d880c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41872/0 (socket says 172.21.15.16:41872) 2024-08-17T02:48:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.809+0000 7f7d8ca00640 1 -- 172.21.15.16:0/1515389494 learned_addr learned my addr 172.21.15.16:0/1515389494 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:39.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.809+0000 7f7d87e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7d881401b0 0x7f7d880b0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45244/0 (socket says 172.21.15.16:45244) 2024-08-17T02:48:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.809+0000 7f7d87400640 1 --2- 172.21.15.16:0/1515389494 >> v2:172.21.15.142:3300/0 conn(0x7f7d880b1440 0x7f7d880caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.809+0000 7f7d8ca00640 1 -- 172.21.15.16:0/1515389494 >> v2:172.21.15.142:3300/0 conn(0x7f7d880b1440 msgr2=0x7f7d880caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.809+0000 7f7d8ca00640 1 --2- 172.21.15.16:0/1515389494 >> v2:172.21.15.142:3300/0 conn(0x7f7d880b1440 0x7f7d880caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.809+0000 7f7d8ca00640 1 -- 172.21.15.16:0/1515389494 >> v1:172.21.15.16:6789/0 conn(0x7f7d881401b0 legacy=0x7f7d880b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.809+0000 7f7d8ca00640 1 -- 172.21.15.16:0/1515389494 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d880ca150 con 0x7f7d88140580 2024-08-17T02:48:39.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.810+0000 7f7d8ca00640 1 --2- 172.21.15.16:0/1515389494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d88140580 0x7f7d880c9040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7d8806aa80 tx=0x7f7d80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7107e84db856a6a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.810+0000 7f7d86a00640 1 -- 172.21.15.16:0/1515389494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80032400 con 0x7f7d88140580 2024-08-17T02:48:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.810+0000 7f7d86a00640 1 -- 172.21.15.16:0/1515389494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8002e050 con 0x7f7d88140580 2024-08-17T02:48:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.810+0000 7f7d86a00640 1 -- 172.21.15.16:0/1515389494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d800327f0 con 0x7f7d88140580 2024-08-17T02:48:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.811+0000 7f7d8ea00640 1 -- 172.21.15.16:0/1515389494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d88140580 msgr2=0x7f7d880c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.811+0000 7f7d8ea00640 1 --2- 172.21.15.16:0/1515389494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d88140580 0x7f7d880c9040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7d8806aa80 tx=0x7f7d80009840 comp rx=0 tx=0).stop 2024-08-17T02:48:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.811+0000 7f7d8ea00640 1 -- 172.21.15.16:0/1515389494 shutdown_connections 2024-08-17T02:48:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.811+0000 7f7d8ea00640 1 --2- 172.21.15.16:0/1515389494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d88140580 0x7f7d880c9040 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.811+0000 7f7d8ea00640 1 --2- 172.21.15.16:0/1515389494 >> v2:172.21.15.142:3300/0 conn(0x7f7d880b1440 0x7f7d880caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.811+0000 7f7d8ea00640 1 -- 172.21.15.16:0/1515389494 >> 172.21.15.16:0/1515389494 conn(0x7f7d880cb9b0 msgr2=0x7f7d880cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.811+0000 7f7d8ea00640 1 -- 172.21.15.16:0/1515389494 shutdown_connections 2024-08-17T02:48:39.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.811+0000 7f7d8ea00640 1 -- 172.21.15.16:0/1515389494 wait complete. 2024-08-17T02:48:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.812+0000 7f7d8ea00640 1 Processor -- start 2024-08-17T02:48:39.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.813+0000 7f7d8ea00640 1 -- start start 2024-08-17T02:48:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.813+0000 7f7d8ea00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d880b1440 0x7f7d881561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.813+0000 7f7d8ea00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d881401b0 0x7f7d880af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.813+0000 7f7d8ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d88140580 0x7f7d8815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.813+0000 7f7d8ea00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880b0540 con 0x7f7d881401b0 2024-08-17T02:48:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.813+0000 7f7d8ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880aff80 con 0x7f7d88140580 2024-08-17T02:48:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.813+0000 7f7d8ea00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880b0260 con 0x7f7d880b1440 2024-08-17T02:48:39.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.814+0000 7f7d87400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d881401b0 0x7f7d880af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.814+0000 7f7d87400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d881401b0 0x7f7d880af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41888/0 (socket says 172.21.15.16:41888) 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.814+0000 7f7d87400640 1 -- 172.21.15.16:0/614961714 learned_addr learned my addr 172.21.15.16:0/614961714 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.814+0000 7f7d8ca00640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d88140580 0x7f7d8815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.814+0000 7f7d87e00640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d880b1440 0x7f7d881561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.814+0000 7f7d87400640 1 -- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d880b1440 msgr2=0x7f7d881561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.814+0000 7f7d87400640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d880b1440 0x7f7d881561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.814+0000 7f7d87400640 1 -- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d88140580 msgr2=0x7f7d8815a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.814+0000 7f7d87400640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d88140580 0x7f7d8815a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.814+0000 7f7d87400640 1 -- 172.21.15.16:0/614961714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d80009ae0 con 0x7f7d881401b0 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.815+0000 7f7d87400640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d881401b0 0x7f7d880af4e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400d890 tx=0x7f7d74008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.815+0000 7f7d7f400640 1 -- 172.21.15.16:0/614961714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400de20 con 0x7f7d881401b0 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.815+0000 7f7d7f400640 1 -- 172.21.15.16:0/614961714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d740098b0 con 0x7f7d881401b0 2024-08-17T02:48:39.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.815+0000 7f7d7f400640 1 -- 172.21.15.16:0/614961714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74009c50 con 0x7f7d881401b0 2024-08-17T02:48:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.815+0000 7f7d8ea00640 1 -- 172.21.15.16:0/614961714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d88157980 con 0x7f7d881401b0 2024-08-17T02:48:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.815+0000 7f7d8ea00640 1 -- 172.21.15.16:0/614961714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d88156730 con 0x7f7d881401b0 2024-08-17T02:48:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.816+0000 7f7d8ea00640 1 -- 172.21.15.16:0/614961714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d88157e00 con 0x7f7d881401b0 2024-08-17T02:48:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.817+0000 7f7d7f400640 1 -- 172.21.15.16:0/614961714 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f7d74014620 con 0x7f7d881401b0 2024-08-17T02:48:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.818+0000 7f7d7f400640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7d68065150 0x7f7d68067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.818+0000 7f7d7f400640 1 -- 172.21.15.16:0/614961714 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f7d74082bc0 con 0x7f7d881401b0 2024-08-17T02:48:39.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.818+0000 7f7d87e00640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7d68065150 0x7f7d68067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:39.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.819+0000 7f7d87e00640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7d68065150 0x7f7d68067600 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7d78007800 tx=0x7f7d78007e20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:39.822+0000 7f7d7f400640 1 -- 172.21.15.16:0/614961714 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7d74050cb0 con 0x7f7d881401b0 2024-08-17T02:48:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.010+0000 7f7d8ea00640 1 -- 172.21.15.16:0/614961714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d88064db0 con 0x7f7d881401b0 2024-08-17T02:48:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.011+0000 7f7d7f400640 1 -- 172.21.15.16:0/614961714 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+15510 (secure 0 0 0) 0x7f7d740542d0 con 0x7f7d881401b0 2024-08-17T02:48:40.012 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:40.013 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":43,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:48:39.616469+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000521811.meta","create_time":"2024-08-17T02:46:27.102528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000521811"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000521811.data","create_time":"2024-08-17T02:46:27.975177+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000521811"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/219684854":"2024-08-18T02:48:36.607854+0000","172.21.15.16:6833/219684854":"2024-08-18T02:48:36.607854+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:48:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.013+0000 7f7d8ea00640 1 -- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7d68065150 msgr2=0x7f7d68067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.013+0000 7f7d8ea00640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7d68065150 0x7f7d68067600 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7d78007800 tx=0x7f7d78007e20 comp rx=0 tx=0).stop 2024-08-17T02:48:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.013+0000 7f7d8ea00640 1 -- 172.21.15.16:0/614961714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d881401b0 msgr2=0x7f7d880af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.013+0000 7f7d8ea00640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d881401b0 0x7f7d880af4e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400d890 tx=0x7f7d74008040 comp rx=0 tx=0).stop 2024-08-17T02:48:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.014+0000 7f7d8ea00640 1 -- 172.21.15.16:0/614961714 shutdown_connections 2024-08-17T02:48:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.014+0000 7f7d8ea00640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7d68065150 0x7f7d68067600 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.014+0000 7f7d8ea00640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d88140580 0x7f7d8815a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.014+0000 7f7d8ea00640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d881401b0 0x7f7d880af4e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.014+0000 7f7d8ea00640 1 --2- 172.21.15.16:0/614961714 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7d880b1440 0x7f7d881561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.014+0000 7f7d8ea00640 1 -- 172.21.15.16:0/614961714 >> 172.21.15.16:0/614961714 conn(0x7f7d880cb9b0 msgr2=0x7f7d880c09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.014+0000 7f7d8ea00640 1 -- 172.21.15.16:0/614961714 shutdown_connections 2024-08-17T02:48:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.014+0000 7f7d8ea00640 1 -- 172.21.15.16:0/614961714 wait complete. 2024-08-17T02:48:40.031 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-17T02:45:14.291903+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs.volume_0000000000521811.meta', 'create_time': '2024-08-17T02:46:27.102528+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000521811'}}}, {'pool': 7, 'pool_name': 'cephfs.volume_0000000000521811.data', 'create_time': '2024-08-17T02:46:27.975177+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000521811'}}}] 2024-08-17T02:48:40.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-17T02:48:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.184+0000 7fc915600640 1 Processor -- start 2024-08-17T02:48:40.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.185+0000 7fc915600640 1 -- start start 2024-08-17T02:48:40.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.186+0000 7fc915600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc910158bd0 0x7fc9100a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.186+0000 7fc915600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fc910158fa0 0x7fc9100aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.186+0000 7fc915600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9100a9ec0 con 0x7fc9100aba20 2024-08-17T02:48:40.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.186+0000 7fc915600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9100aa1a0 con 0x7fc910158bd0 2024-08-17T02:48:40.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.186+0000 7fc915600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9100aa480 con 0x7fc910158fa0 2024-08-17T02:48:40.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.186+0000 7fc90ea00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc910158bd0 0x7fc9100a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.186+0000 7fc90f400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc9100aba20 0x7fc9100ad790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45246/0 (socket says 172.21.15.16:45246) 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.186+0000 7fc90f400640 1 -- 172.21.15.16:0/3401987478 learned_addr learned my addr 172.21.15.16:0/3401987478 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.186+0000 7fc907e00640 1 --2- 172.21.15.16:0/3401987478 >> v2:172.21.15.142:3300/0 conn(0x7fc910158fa0 0x7fc9100aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.187+0000 7fc90ea00640 1 -- 172.21.15.16:0/3401987478 >> v2:172.21.15.142:3300/0 conn(0x7fc910158fa0 msgr2=0x7fc9100aaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.187+0000 7fc90ea00640 1 --2- 172.21.15.16:0/3401987478 >> v2:172.21.15.142:3300/0 conn(0x7fc910158fa0 0x7fc9100aaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.187+0000 7fc90ea00640 1 -- 172.21.15.16:0/3401987478 >> v1:172.21.15.16:6789/0 conn(0x7fc9100aba20 legacy=0x7fc9100ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.187+0000 7fc90ea00640 1 -- 172.21.15.16:0/3401987478 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9100aa760 con 0x7fc910158bd0 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.187+0000 7fc90e000640 1 -- 172.21.15.16:0/3401987478 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040124517 0 0) 0x7fc9100a9ec0 con 0x7fc9100aba20 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.188+0000 7fc90ea00640 1 --2- 172.21.15.16:0/3401987478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910158bd0 0x7fc9100a98e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc9100af6e0 tx=0x7fc9000090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae15c5ab7674a5d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.188+0000 7fc90e000640 1 -- 172.21.15.16:0/3401987478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90002adb0 con 0x7fc910158bd0 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.188+0000 7fc90e000640 1 -- 172.21.15.16:0/3401987478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc900032440 con 0x7fc910158bd0 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.188+0000 7fc90e000640 1 -- 172.21.15.16:0/3401987478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000327e0 con 0x7fc910158bd0 2024-08-17T02:48:40.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.188+0000 7fc915600640 1 -- 172.21.15.16:0/3401987478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910158bd0 msgr2=0x7fc9100a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.188+0000 7fc915600640 1 --2- 172.21.15.16:0/3401987478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910158bd0 0x7fc9100a98e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc9100af6e0 tx=0x7fc9000090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.189+0000 7fc915600640 1 -- 172.21.15.16:0/3401987478 shutdown_connections 2024-08-17T02:48:40.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.189+0000 7fc915600640 1 --2- 172.21.15.16:0/3401987478 >> v2:172.21.15.142:3300/0 conn(0x7fc910158fa0 0x7fc9100aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.189+0000 7fc915600640 1 --2- 172.21.15.16:0/3401987478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910158bd0 0x7fc9100a98e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.189+0000 7fc915600640 1 -- 172.21.15.16:0/3401987478 >> 172.21.15.16:0/3401987478 conn(0x7fc910066cd0 msgr2=0x7fc9100670d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:40.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.189+0000 7fc915600640 1 -- 172.21.15.16:0/3401987478 shutdown_connections 2024-08-17T02:48:40.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.189+0000 7fc915600640 1 -- 172.21.15.16:0/3401987478 wait complete. 2024-08-17T02:48:40.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.190+0000 7fc915600640 1 Processor -- start 2024-08-17T02:48:40.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.190+0000 7fc915600640 1 -- start start 2024-08-17T02:48:40.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.191+0000 7fc915600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9100aba20 0x7fc9100c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.191+0000 7fc915600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910158bd0 0x7fc9100c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.191+0000 7fc915600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc910158fa0 0x7fc9100c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.191+0000 7fc915600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100c5d60 con 0x7fc910158bd0 2024-08-17T02:48:40.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.191+0000 7fc915600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100c5150 con 0x7fc910158fa0 2024-08-17T02:48:40.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.191+0000 7fc915600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100c5430 con 0x7fc9100aba20 2024-08-17T02:48:40.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.191+0000 7fc907e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910158bd0 0x7fc9100c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.191+0000 7fc907e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910158bd0 0x7fc9100c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41896/0 (socket says 172.21.15.16:41896) 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.191+0000 7fc907e00640 1 -- 172.21.15.16:0/1898432626 learned_addr learned my addr 172.21.15.16:0/1898432626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc90ea00640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9100aba20 0x7fc9100c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc90f400640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc910158fa0 0x7fc9100c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc907e00640 1 -- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9100aba20 msgr2=0x7fc9100c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc907e00640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9100aba20 0x7fc9100c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc907e00640 1 -- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc910158fa0 msgr2=0x7fc9100c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc907e00640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc910158fa0 0x7fc9100c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc907e00640 1 -- 172.21.15.16:0/1898432626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc900009390 con 0x7fc910158bd0 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc90ea00640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9100aba20 0x7fc9100c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc90f400640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc910158fa0 0x7fc9100c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc907e00640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910158bd0 0x7fc9100c6700 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800d860 tx=0x7fc8f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc906000640 1 -- 172.21.15.16:0/1898432626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f80096a0 con 0x7fc910158bd0 2024-08-17T02:48:40.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc906000640 1 -- 172.21.15.16:0/1898432626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f8009800 con 0x7fc910158bd0 2024-08-17T02:48:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc906000640 1 -- 172.21.15.16:0/1898432626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8009bc0 con 0x7fc910158bd0 2024-08-17T02:48:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc915600640 1 -- 172.21.15.16:0/1898432626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9100c5710 con 0x7fc910158bd0 2024-08-17T02:48:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.192+0000 7fc915600640 1 -- 172.21.15.16:0/1898432626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9100c3f30 con 0x7fc910158bd0 2024-08-17T02:48:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.193+0000 7fc915600640 1 -- 172.21.15.16:0/1898432626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9100c5b90 con 0x7fc910158bd0 2024-08-17T02:48:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.194+0000 7fc906000640 1 -- 172.21.15.16:0/1898432626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fc8f8014410 con 0x7fc910158bd0 2024-08-17T02:48:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.195+0000 7fc906000640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc8ec065150 0x7fc8ec067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.195+0000 7fc906000640 1 -- 172.21.15.16:0/1898432626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fc8f8082d00 con 0x7fc910158bd0 2024-08-17T02:48:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.195+0000 7fc90ea00640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc8ec065150 0x7fc8ec067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.196+0000 7fc90ea00640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc8ec065150 0x7fc8ec067600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc900034000 tx=0x7fc900032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:40.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.199+0000 7fc906000640 1 -- 172.21.15.16:0/1898432626 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc8f8050d60 con 0x7fc910158bd0 2024-08-17T02:48:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.382+0000 7fc915600640 1 -- 172.21.15.16:0/1898432626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc910064db0 con 0x7fc910158bd0 2024-08-17T02:48:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.383+0000 7fc906000640 1 -- 172.21.15.16:0/1898432626 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8f8054380 con 0x7fc910158bd0 2024-08-17T02:48:40.384 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-08-17T02:48:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.385+0000 7fc915600640 1 -- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc8ec065150 msgr2=0x7fc8ec067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.385+0000 7fc915600640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc8ec065150 0x7fc8ec067600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc900034000 tx=0x7fc900032000 comp rx=0 tx=0).stop 2024-08-17T02:48:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.385+0000 7fc915600640 1 -- 172.21.15.16:0/1898432626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910158bd0 msgr2=0x7fc9100c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.385+0000 7fc915600640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910158bd0 0x7fc9100c6700 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800d860 tx=0x7fc8f8008040 comp rx=0 tx=0).stop 2024-08-17T02:48:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.385+0000 7fc915600640 1 -- 172.21.15.16:0/1898432626 shutdown_connections 2024-08-17T02:48:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.385+0000 7fc915600640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc8ec065150 0x7fc8ec067600 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.385+0000 7fc915600640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc910158fa0 0x7fc9100c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.385+0000 7fc915600640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc910158bd0 0x7fc9100c6700 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.385+0000 7fc915600640 1 --2- 172.21.15.16:0/1898432626 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc9100aba20 0x7fc9100c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.385+0000 7fc915600640 1 -- 172.21.15.16:0/1898432626 >> 172.21.15.16:0/1898432626 conn(0x7fc910066cd0 msgr2=0x7fc9100acd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.385+0000 7fc915600640 1 -- 172.21.15.16:0/1898432626 shutdown_connections 2024-08-17T02:48:40.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.385+0000 7fc915600640 1 -- 172.21.15.16:0/1898432626 wait complete. 2024-08-17T02:48:40.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000521811.meta pg_num 2024-08-17T02:48:40.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.554+0000 7f6f22800640 1 Processor -- start 2024-08-17T02:48:40.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.554+0000 7f6f22800640 1 -- start start 2024-08-17T02:48:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.555+0000 7f6f22800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f1c156c50 0x7f6f1c0c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.555+0000 7f6f22800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0c8bd0 0x7f6f1c0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.555+0000 7f6f22800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1c0c9570 con 0x7f6f1c0adcf0 2024-08-17T02:48:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.555+0000 7f6f22800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f1c0ab130 con 0x7f6f1c156c50 2024-08-17T02:48:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.555+0000 7f6f22800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0ae5a0 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.555+0000 7f6f1b400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f1c156c50 0x7f6f1c0c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.555+0000 7f6f1b400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f1c156c50 0x7f6f1c0c7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41906/0 (socket says 172.21.15.16:41906) 2024-08-17T02:48:40.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.555+0000 7f6f1b400640 1 -- 172.21.15.16:0/1572250130 learned_addr learned my addr 172.21.15.16:0/1572250130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.556+0000 7f6f1be00640 1 --2- 172.21.15.16:0/1572250130 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0c8bd0 0x7f6f1c0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.556+0000 7f6f1b400640 1 -- 172.21.15.16:0/1572250130 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0c8bd0 msgr2=0x7f6f1c0c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.556+0000 7f6f1b400640 1 --2- 172.21.15.16:0/1572250130 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0c8bd0 0x7f6f1c0c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.556+0000 7f6f1b400640 1 -- 172.21.15.16:0/1572250130 >> v1:172.21.15.142:6789/0 conn(0x7f6f1c0adcf0 legacy=0x7f6f1c0ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.556+0000 7f6f1b400640 1 -- 172.21.15.16:0/1572250130 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c0ae2c0 con 0x7f6f1c156c50 2024-08-17T02:48:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.556+0000 7f6f1be00640 1 --2- 172.21.15.16:0/1572250130 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0c8bd0 0x7f6f1c0c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:40.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.557+0000 7f6f1b400640 1 --2- 172.21.15.16:0/1572250130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f1c156c50 0x7f6f1c0c7cf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009910 tx=0x7f6f0c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4dcbad47b80ef5 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.557+0000 7f6f1aa00640 1 -- 172.21.15.16:0/1572250130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c02ac90 con 0x7f6f1c156c50 2024-08-17T02:48:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.557+0000 7f6f1aa00640 1 -- 172.21.15.16:0/1572250130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c02adf0 con 0x7f6f1c156c50 2024-08-17T02:48:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.557+0000 7f6f1aa00640 1 -- 172.21.15.16:0/1572250130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c032670 con 0x7f6f1c156c50 2024-08-17T02:48:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.558+0000 7f6f22800640 1 -- 172.21.15.16:0/1572250130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f1c156c50 msgr2=0x7f6f1c0c7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.558+0000 7f6f22800640 1 --2- 172.21.15.16:0/1572250130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f1c156c50 0x7f6f1c0c7cf0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009910 tx=0x7f6f0c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.558+0000 7f6f22800640 1 -- 172.21.15.16:0/1572250130 shutdown_connections 2024-08-17T02:48:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.558+0000 7f6f22800640 1 --2- 172.21.15.16:0/1572250130 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0c8bd0 0x7f6f1c0c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.558+0000 7f6f22800640 1 --2- 172.21.15.16:0/1572250130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f1c156c50 0x7f6f1c0c7cf0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.558+0000 7f6f22800640 1 -- 172.21.15.16:0/1572250130 >> 172.21.15.16:0/1572250130 conn(0x7f6f1c067750 msgr2=0x7f6f1c0c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.558+0000 7f6f22800640 1 -- 172.21.15.16:0/1572250130 shutdown_connections 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.558+0000 7f6f22800640 1 -- 172.21.15.16:0/1572250130 wait complete. 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.559+0000 7f6f22800640 1 Processor -- start 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.560+0000 7f6f22800640 1 -- start start 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.560+0000 7f6f22800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0adcf0 0x7f6f1c169710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.560+0000 7f6f22800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f1c0c8bd0 0x7f6f1c16a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.560+0000 7f6f22800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f1c156c50 0x7f6f1c155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.560+0000 7f6f22800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c1570b0 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.560+0000 7f6f22800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c169d30 con 0x7f6f1c156c50 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.560+0000 7f6f22800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c16a010 con 0x7f6f1c0adcf0 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.561+0000 7f6f13e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f1c0c8bd0 0x7f6f1c16a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.561+0000 7f6f13e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f1c0c8bd0 0x7f6f1c16a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41922/0 (socket says 172.21.15.16:41922) 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.561+0000 7f6f13e00640 1 -- 172.21.15.16:0/3684551607 learned_addr learned my addr 172.21.15.16:0/3684551607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.561+0000 7f6f1be00640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f1c156c50 0x7f6f1c155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.561+0000 7f6f1b400640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0adcf0 0x7f6f1c169710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.561+0000 7f6f13e00640 1 -- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0adcf0 msgr2=0x7f6f1c169710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.561+0000 7f6f13e00640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0adcf0 0x7f6f1c169710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.561+0000 7f6f13e00640 1 -- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f1c156c50 msgr2=0x7f6f1c155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.561+0000 7f6f13e00640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f1c156c50 0x7f6f1c155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.561+0000 7f6f13e00640 1 -- 172.21.15.16:0/3684551607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c009450 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.561+0000 7f6f1b400640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0adcf0 0x7f6f1c169710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.561+0000 7f6f1be00640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f1c156c50 0x7f6f1c155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.562+0000 7f6f13e00640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f1c0c8bd0 0x7f6f1c16a920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400c5e0 tx=0x7f6f04007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.562+0000 7f6f12a00640 1 -- 172.21.15.16:0/3684551607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04012630 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.562+0000 7f6f12a00640 1 -- 172.21.15.16:0/3684551607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f040127b0 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.562+0000 7f6f12a00640 1 -- 172.21.15.16:0/3684551607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04012bb0 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.562+0000 7f6f22800640 1 -- 172.21.15.16:0/3684551607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1c16a2f0 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.562+0000 7f6f22800640 1 -- 172.21.15.16:0/3684551607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1c16ae60 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.563+0000 7f6f22800640 1 -- 172.21.15.16:0/3684551607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1c071050 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.564+0000 7f6f12a00640 1 -- 172.21.15.16:0/3684551607 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f6f0401b400 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.565+0000 7f6f12a00640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6ef8065180 0x7f6ef8067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.565+0000 7f6f12a00640 1 -- 172.21.15.16:0/3684551607 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f6f040826a0 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.565+0000 7f6f1b400640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6ef8065180 0x7f6ef8067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.566+0000 7f6f1b400640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6ef8065180 0x7f6ef8067630 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c034000 tx=0x7f6f0c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.569+0000 7f6f12a00640 1 -- 172.21.15.16:0/3684551607 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f6f04050640 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.752+0000 7f6f22800640 1 -- 172.21.15.16:0/3684551607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000521811.meta", "var": "pg_num"} v 0) v1 -- 0x7f6f1c0ac230 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.752+0000 7f6f12a00640 1 -- 172.21.15.16:0/3684551607 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000521811.meta", "var": "pg_num"}]=0 v43) v1 ==== 124+0+11 (secure 0 0 0) 0x7f6f04053c60 con 0x7f6f1c0c8bd0 2024-08-17T02:48:40.754 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:48:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.754+0000 7f6f22800640 1 -- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6ef8065180 msgr2=0x7f6ef8067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.754+0000 7f6f22800640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6ef8065180 0x7f6ef8067630 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c034000 tx=0x7f6f0c032000 comp rx=0 tx=0).stop 2024-08-17T02:48:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.754+0000 7f6f22800640 1 -- 172.21.15.16:0/3684551607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f1c0c8bd0 msgr2=0x7f6f1c16a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.754+0000 7f6f22800640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f1c0c8bd0 0x7f6f1c16a920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400c5e0 tx=0x7f6f04007ed0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.755+0000 7f6f22800640 1 -- 172.21.15.16:0/3684551607 shutdown_connections 2024-08-17T02:48:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.755+0000 7f6f22800640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f6ef8065180 0x7f6ef8067630 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.755+0000 7f6f22800640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f1c156c50 0x7f6f1c155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.755+0000 7f6f22800640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f1c0c8bd0 0x7f6f1c16a920 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.755+0000 7f6f22800640 1 --2- 172.21.15.16:0/3684551607 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f6f1c0adcf0 0x7f6f1c169710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.755+0000 7f6f22800640 1 -- 172.21.15.16:0/3684551607 >> 172.21.15.16:0/3684551607 conn(0x7f6f1c067750 msgr2=0x7f6f1c06d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:40.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.755+0000 7f6f22800640 1 -- 172.21.15.16:0/3684551607 shutdown_connections 2024-08-17T02:48:40.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.755+0000 7f6f22800640 1 -- 172.21.15.16:0/3684551607 wait complete. 2024-08-17T02:48:40.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000521811.data pg_num 2024-08-17T02:48:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.924+0000 7fd826000640 1 Processor -- start 2024-08-17T02:48:40.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.924+0000 7fd826000640 1 -- start start 2024-08-17T02:48:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd826000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd82015be50 0x7fd8200ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd826000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8200aec90 0x7fd8200aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd826000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fd8200ab3b0 0x7fd8200aace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd826000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8200b05b0 con 0x7fd8200aec90 2024-08-17T02:48:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd826000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd82006d4a0 con 0x7fd8200ab3b0 2024-08-17T02:48:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd826000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82006c890 con 0x7fd82015be50 2024-08-17T02:48:40.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd81fe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8200aec90 0x7fd8200aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd81fe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8200aec90 0x7fd8200aa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41928/0 (socket says 172.21.15.16:41928) 2024-08-17T02:48:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd81fe00640 1 -- 172.21.15.16:0/2849780238 learned_addr learned my addr 172.21.15.16:0/2849780238 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd81fe00640 1 -- 172.21.15.16:0/2849780238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd82015be50 msgr2=0x7fd8200ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd825600640 1 --2- 172.21.15.16:0/2849780238 >> v2:172.21.15.142:3300/0 conn(0x7fd8200ab3b0 0x7fd8200aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd824c00640 1 --2- 172.21.15.16:0/2849780238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd82015be50 0x7fd8200ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.926+0000 7fd81fe00640 1 --2- 172.21.15.16:0/2849780238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd82015be50 0x7fd8200ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.927+0000 7fd81fe00640 1 -- 172.21.15.16:0/2849780238 >> v2:172.21.15.142:3300/0 conn(0x7fd8200ab3b0 msgr2=0x7fd8200aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.927+0000 7fd81fe00640 1 --2- 172.21.15.16:0/2849780238 >> v2:172.21.15.142:3300/0 conn(0x7fd8200ab3b0 0x7fd8200aace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.927+0000 7fd81fe00640 1 -- 172.21.15.16:0/2849780238 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82006cb70 con 0x7fd8200aec90 2024-08-17T02:48:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.927+0000 7fd825600640 1 --2- 172.21.15.16:0/2849780238 >> v2:172.21.15.142:3300/0 conn(0x7fd8200ab3b0 0x7fd8200aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:40.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.927+0000 7fd824c00640 1 --2- 172.21.15.16:0/2849780238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd82015be50 0x7fd8200ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:40.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.928+0000 7fd81fe00640 1 --2- 172.21.15.16:0/2849780238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8200aec90 0x7fd8200aa7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd820064320 tx=0x7fd80c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b22e287b2269fe server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.928+0000 7fd81f400640 1 -- 172.21.15.16:0/2849780238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c02add0 con 0x7fd8200aec90 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.928+0000 7fd81f400640 1 -- 172.21.15.16:0/2849780238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c032440 con 0x7fd8200aec90 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.928+0000 7fd81f400640 1 -- 172.21.15.16:0/2849780238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c032820 con 0x7fd8200aec90 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.928+0000 7fd826000640 1 -- 172.21.15.16:0/2849780238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8200aec90 msgr2=0x7fd8200aa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.928+0000 7fd826000640 1 --2- 172.21.15.16:0/2849780238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8200aec90 0x7fd8200aa7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd820064320 tx=0x7fd80c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.928+0000 7fd826000640 1 -- 172.21.15.16:0/2849780238 shutdown_connections 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.928+0000 7fd826000640 1 --2- 172.21.15.16:0/2849780238 >> v2:172.21.15.142:3300/0 conn(0x7fd8200ab3b0 0x7fd8200aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.928+0000 7fd826000640 1 --2- 172.21.15.16:0/2849780238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8200aec90 0x7fd8200aa7a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.928+0000 7fd826000640 1 --2- 172.21.15.16:0/2849780238 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd82015be50 0x7fd8200ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.928+0000 7fd826000640 1 -- 172.21.15.16:0/2849780238 >> 172.21.15.16:0/2849780238 conn(0x7fd820066cd0 msgr2=0x7fd8200670d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.929+0000 7fd826000640 1 -- 172.21.15.16:0/2849780238 shutdown_connections 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.929+0000 7fd826000640 1 -- 172.21.15.16:0/2849780238 wait complete. 2024-08-17T02:48:40.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.930+0000 7fd826000640 1 Processor -- start 2024-08-17T02:48:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.930+0000 7fd826000640 1 -- start start 2024-08-17T02:48:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.931+0000 7fd826000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8200ab3b0 0x7fd820157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.931+0000 7fd826000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8200aec90 0x7fd820168a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.931+0000 7fd826000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82015be50 0x7fd82016c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.931+0000 7fd826000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd820140af0 con 0x7fd8200ab3b0 2024-08-17T02:48:40.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.931+0000 7fd826000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd820159750 con 0x7fd82015be50 2024-08-17T02:48:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.931+0000 7fd826000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8201581a0 con 0x7fd8200aec90 2024-08-17T02:48:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.931+0000 7fd824c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8200ab3b0 0x7fd820157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.931+0000 7fd824c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8200ab3b0 0x7fd820157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41936/0 (socket says 172.21.15.16:41936) 2024-08-17T02:48:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.931+0000 7fd824c00640 1 -- 172.21.15.16:0/2517522534 learned_addr learned my addr 172.21.15.16:0/2517522534 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.931+0000 7fd81fe00640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8200aec90 0x7fd820168a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.931+0000 7fd825600640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82015be50 0x7fd82016c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.932+0000 7fd824c00640 1 -- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8200aec90 msgr2=0x7fd820168a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.932+0000 7fd824c00640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8200aec90 0x7fd820168a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.932+0000 7fd824c00640 1 -- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82015be50 msgr2=0x7fd82016c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.932+0000 7fd824c00640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82015be50 0x7fd82016c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:40.936 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.932+0000 7fd824c00640 1 -- 172.21.15.16:0/2517522534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c009450 con 0x7fd8200ab3b0 2024-08-17T02:48:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.932+0000 7fd825600640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82015be50 0x7fd82016c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.932+0000 7fd824c00640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8200ab3b0 0x7fd820157c60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd81400c880 tx=0x7fd814008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.932+0000 7fd813e00640 1 -- 172.21.15.16:0/2517522534 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81400ce10 con 0x7fd8200ab3b0 2024-08-17T02:48:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.932+0000 7fd813e00640 1 -- 172.21.15.16:0/2517522534 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd814007890 con 0x7fd8200ab3b0 2024-08-17T02:48:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.932+0000 7fd826000640 1 -- 172.21.15.16:0/2517522534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd820152670 con 0x7fd8200ab3b0 2024-08-17T02:48:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.932+0000 7fd813e00640 1 -- 172.21.15.16:0/2517522534 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814007c10 con 0x7fd8200ab3b0 2024-08-17T02:48:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.932+0000 7fd826000640 1 -- 172.21.15.16:0/2517522534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd820159cd0 con 0x7fd8200ab3b0 2024-08-17T02:48:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.934+0000 7fd826000640 1 -- 172.21.15.16:0/2517522534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd820071050 con 0x7fd8200ab3b0 2024-08-17T02:48:40.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.934+0000 7fd813e00640 1 -- 172.21.15.16:0/2517522534 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fd8140113f0 con 0x7fd8200ab3b0 2024-08-17T02:48:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.935+0000 7fd813e00640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd800065150 0x7fd800067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.935+0000 7fd813e00640 1 -- 172.21.15.16:0/2517522534 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fd8140823e0 con 0x7fd8200ab3b0 2024-08-17T02:48:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.935+0000 7fd81fe00640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd800065150 0x7fd800067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.936+0000 7fd81fe00640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd800065150 0x7fd800067600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd80c034000 tx=0x7fd80c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:40.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:40.939+0000 7fd813e00640 1 -- 172.21.15.16:0/2517522534 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fd814053f60 con 0x7fd8200ab3b0 2024-08-17T02:48:41.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.123+0000 7fd826000640 1 -- 172.21.15.16:0/2517522534 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000521811.data", "var": "pg_num"} v 0) v1 -- 0x7fd8200aa4f0 con 0x7fd8200ab3b0 2024-08-17T02:48:41.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.124+0000 7fd813e00640 1 -- 172.21.15.16:0/2517522534 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000521811.data", "var": "pg_num"}]=0 v43) v1 ==== 124+0+11 (secure 0 0 0) 0x7fd814053af0 con 0x7fd8200ab3b0 2024-08-17T02:48:41.126 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-08-17T02:48:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.126+0000 7fd826000640 1 -- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd800065150 msgr2=0x7fd800067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.126+0000 7fd826000640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd800065150 0x7fd800067600 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd80c034000 tx=0x7fd80c032000 comp rx=0 tx=0).stop 2024-08-17T02:48:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.126+0000 7fd826000640 1 -- 172.21.15.16:0/2517522534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8200ab3b0 msgr2=0x7fd820157c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.126+0000 7fd826000640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8200ab3b0 0x7fd820157c60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd81400c880 tx=0x7fd814008040 comp rx=0 tx=0).stop 2024-08-17T02:48:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.126+0000 7fd826000640 1 -- 172.21.15.16:0/2517522534 shutdown_connections 2024-08-17T02:48:41.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.126+0000 7fd826000640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fd800065150 0x7fd800067600 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.126+0000 7fd826000640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82015be50 0x7fd82016c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.126+0000 7fd826000640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fd8200aec90 0x7fd820168a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.126+0000 7fd826000640 1 --2- 172.21.15.16:0/2517522534 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8200ab3b0 0x7fd820157c60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.126+0000 7fd826000640 1 -- 172.21.15.16:0/2517522534 >> 172.21.15.16:0/2517522534 conn(0x7fd820066cd0 msgr2=0x7fd8200ad1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.126+0000 7fd826000640 1 -- 172.21.15.16:0/2517522534 shutdown_connections 2024-08-17T02:48:41.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.126+0000 7fd826000640 1 -- 172.21.15.16:0/2517522534 wait complete. 2024-08-17T02:48:41.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:48:41.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.293+0000 7f7dd9400640 1 Processor -- start 2024-08-17T02:48:41.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.293+0000 7f7dd9400640 1 -- start start 2024-08-17T02:48:41.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.294+0000 7f7dd9400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4155180 0x7f7dd40aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.294+0000 7f7dd9400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f7dd406e080 0x7f7dd406e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.294+0000 7f7dd9400640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd40afc10 con 0x7f7dd4155550 2024-08-17T02:48:41.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.294+0000 7f7dd9400640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd406ea20 con 0x7f7dd406e080 2024-08-17T02:48:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.294+0000 7f7dd9400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40af000 con 0x7f7dd4155180 2024-08-17T02:48:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7dd4155550 0x7f7dd40ae070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45260/0 (socket says 172.21.15.16:45260) 2024-08-17T02:48:41.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1e00640 1 -- 172.21.15.16:0/2403663605 learned_addr learned my addr 172.21.15.16:0/2403663605 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd3200640 1 --2- 172.21.15.16:0/2403663605 >> v2:172.21.15.142:3300/0 conn(0x7f7dd406e080 0x7f7dd406e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd2800640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4155180 0x7f7dd40aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042265339 0 0) 0x7f7dd40afc10 con 0x7f7dd4155550 2024-08-17T02:48:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7db80034e0 con 0x7f7dd4155550 2024-08-17T02:48:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3780396465 0 0) 0x7f7db80034e0 con 0x7f7dd4155550 2024-08-17T02:48:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4155180 msgr2=0x7f7dd40aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1400640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4155180 0x7f7dd40aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 >> v2:172.21.15.142:3300/0 conn(0x7f7dd406e080 msgr2=0x7f7dd406e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1400640 1 --2- 172.21.15.16:0/2403663605 >> v2:172.21.15.142:3300/0 conn(0x7f7dd406e080 0x7f7dd406e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd40af2e0 con 0x7f7dd4155550 2024-08-17T02:48:41.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1567558159 0 0) 0x7f7dc0001490 con 0x7f7dd4155550 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 >> v1:172.21.15.16:6789/0 conn(0x7f7dd4155550 legacy=0x7f7dd40ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.295+0000 7f7dd1400640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db8003a50 0x7f7db8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd1400640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db8004780 0x7f7db8004bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd1400640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4155180 0x7f7db8005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd406ea20 con 0x7f7db8003a50 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40af000 con 0x7f7db8004780 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db800ba80 con 0x7f7dd4155180 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd2800640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db8003a50 0x7f7db8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd3200640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4155180 0x7f7db8005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd1e00640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db8004780 0x7f7db8004bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd2800640 1 -- 172.21.15.16:0/2403663605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4155180 msgr2=0x7f7db8005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd2800640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7dd4155180 0x7f7db8005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd2800640 1 -- 172.21.15.16:0/2403663605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db8004780 msgr2=0x7f7db8004bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd2800640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db8004780 0x7f7db8004bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd2800640 1 -- 172.21.15.16:0/2403663605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db80034e0 con 0x7f7db8003a50 2024-08-17T02:48:41.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.296+0000 7f7dd1e00640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7db8004780 0x7f7db8004bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:41.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.297+0000 7f7dd2800640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db8003a50 0x7f7db8003e60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009b70 tx=0x7f7dc40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34ad5271eb85df7 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:41.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.297+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4031400 con 0x7f7db8003a50 2024-08-17T02:48:41.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.297+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc402d050 con 0x7f7db8003a50 2024-08-17T02:48:41.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.297+0000 7f7dd1400640 1 -- 172.21.15.16:0/2403663605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4031790 con 0x7f7db8003a50 2024-08-17T02:48:41.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.298+0000 7f7dd9400640 1 -- 172.21.15.16:0/2403663605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db8003a50 msgr2=0x7f7db8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.298+0000 7f7dd9400640 1 --2- 172.21.15.16:0/2403663605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7db8003a50 0x7f7db8003e60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009b70 tx=0x7f7dc40090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.298+0000 7f7dd3200640 1 -- 172.21.15.16:0/2403663605 reap_dead start 2024-08-17T02:48:41.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.298+0000 7f7dd9400640 1 -- 172.21.15.16:0/2403663605 shutdown_connections 2024-08-17T02:48:41.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.298+0000 7f7dd9400640 1 -- 172.21.15.16:0/2403663605 >> 172.21.15.16:0/2403663605 conn(0x7f7dd40cb9b0 msgr2=0x7f7dd40cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:41.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.298+0000 7f7dd9400640 1 -- 172.21.15.16:0/2403663605 shutdown_connections 2024-08-17T02:48:41.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.298+0000 7f7dd9400640 1 -- 172.21.15.16:0/2403663605 wait complete. 2024-08-17T02:48:41.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.299+0000 7f7dd9400640 1 Processor -- start 2024-08-17T02:48:41.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.300+0000 7f7dd9400640 1 -- start start 2024-08-17T02:48:41.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.300+0000 7f7dd9400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db8003a50 0x7f7dd40c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.300+0000 7f7dd9400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd4157f00 0x7f7dd40c3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.300+0000 7f7dd9400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd40c3cc0 0x7f7dd4147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.300+0000 7f7dd9400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40c1ea0 con 0x7f7dd4157f00 2024-08-17T02:48:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.300+0000 7f7dd9400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd4158370 con 0x7f7dd40c3cc0 2024-08-17T02:48:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.300+0000 7f7dd9400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40c1290 con 0x7f7db8003a50 2024-08-17T02:48:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.301+0000 7f7dd1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd4157f00 0x7f7dd40c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.301+0000 7f7dd1e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd4157f00 0x7f7dd40c3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41950/0 (socket says 172.21.15.16:41950) 2024-08-17T02:48:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.301+0000 7f7dd1e00640 1 -- 172.21.15.16:0/3039253775 learned_addr learned my addr 172.21.15.16:0/3039253775 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.301+0000 7f7dd3200640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd40c3cc0 0x7f7dd4147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.301+0000 7f7dd2800640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db8003a50 0x7f7dd40c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.301+0000 7f7dd1e00640 1 -- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db8003a50 msgr2=0x7f7dd40c0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.301+0000 7f7dd1e00640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db8003a50 0x7f7dd40c0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.301+0000 7f7dd1e00640 1 -- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd40c3cc0 msgr2=0x7f7dd4147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.301+0000 7f7dd1e00640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd40c3cc0 0x7f7dd4147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.301+0000 7f7dd1e00640 1 -- 172.21.15.16:0/3039253775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd40c1570 con 0x7f7dd4157f00 2024-08-17T02:48:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.301+0000 7f7dd1e00640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd4157f00 0x7f7dd40c3780 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000eb90 tx=0x7f7dc0001450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.302+0000 7f7dca000640 1 -- 172.21.15.16:0/3039253775 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000c820 con 0x7f7dd4157f00 2024-08-17T02:48:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.302+0000 7f7dca000640 1 -- 172.21.15.16:0/3039253775 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc000c980 con 0x7f7dd4157f00 2024-08-17T02:48:41.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.302+0000 7f7dca000640 1 -- 172.21.15.16:0/3039253775 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000cd00 con 0x7f7dd4157f00 2024-08-17T02:48:41.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.302+0000 7f7dd9400640 1 -- 172.21.15.16:0/3039253775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd40c1850 con 0x7f7dd4157f00 2024-08-17T02:48:41.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.302+0000 7f7dd9400640 1 -- 172.21.15.16:0/3039253775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd4148070 con 0x7f7dd4157f00 2024-08-17T02:48:41.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.303+0000 7f7dd9400640 1 -- 172.21.15.16:0/3039253775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd40c2050 con 0x7f7dd4157f00 2024-08-17T02:48:41.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.304+0000 7f7dca000640 1 -- 172.21.15.16:0/3039253775 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f7dc0013630 con 0x7f7dd4157f00 2024-08-17T02:48:41.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.304+0000 7f7dca000640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7db4065180 0x7f7db4067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.305+0000 7f7dca000640 1 -- 172.21.15.16:0/3039253775 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f7dc0082460 con 0x7f7dd4157f00 2024-08-17T02:48:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.305+0000 7f7dd2800640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7db4065180 0x7f7db4067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.306+0000 7f7dd2800640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7db4065180 0x7f7db4067630 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4034000 tx=0x7f7dc4035000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.309+0000 7f7dca000640 1 -- 172.21.15.16:0/3039253775 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f7dd40c2050 con 0x7f7dd4157f00 2024-08-17T02:48:41.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.528+0000 7f7dd9400640 1 -- 172.21.15.16:0/3039253775 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7dd4064db0 con 0x7f7dd4157f00 2024-08-17T02:48:41.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.528+0000 7f7dca000640 1 -- 172.21.15.16:0/3039253775 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4621 (secure 0 0 0) 0x7f7dc0050400 con 0x7f7dd4157f00 2024-08-17T02:48:41.530 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:41.530 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2889208700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2889208700},{"type":"v1","addr":"172.21.15.142:6837","nonce":2889208700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4287834697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4287834697},{"type":"v1","addr":"172.21.15.142:6835","nonce":4287834697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1841153868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1841153868},{"type":"v1","addr":"172.21.15.16:6835","nonce":1841153868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:28.938945+0000","modified":"2024-08-17T02:46:30.943869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6834/3115083427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3115083427},{"type":"v1","addr":"172.21.15.16:6834","nonce":3115083427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000521811","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:48:41.530 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-08-17T02:48:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.530+0000 7f7dd9400640 1 -- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7db4065180 msgr2=0x7f7db4067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.530+0000 7f7dd9400640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7db4065180 0x7f7db4067630 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4034000 tx=0x7f7dc4035000 comp rx=0 tx=0).stop 2024-08-17T02:48:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.530+0000 7f7dd9400640 1 -- 172.21.15.16:0/3039253775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd4157f00 msgr2=0x7f7dd40c3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.530+0000 7f7dd9400640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd4157f00 0x7f7dd40c3780 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7dc000eb90 tx=0x7f7dc0001450 comp rx=0 tx=0).stop 2024-08-17T02:48:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.530+0000 7f7dd9400640 1 -- 172.21.15.16:0/3039253775 shutdown_connections 2024-08-17T02:48:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.530+0000 7f7dd9400640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7db4065180 0x7f7db4067630 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.530+0000 7f7dd9400640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7dd40c3cc0 0x7f7dd4147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.530+0000 7f7dd9400640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7dd4157f00 0x7f7dd40c3780 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.530+0000 7f7dd9400640 1 --2- 172.21.15.16:0/3039253775 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7db8003a50 0x7f7dd40c0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.530+0000 7f7dd9400640 1 -- 172.21.15.16:0/3039253775 >> 172.21.15.16:0/3039253775 conn(0x7f7dd40cb9b0 msgr2=0x7f7dd40ab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.531+0000 7f7dd9400640 1 -- 172.21.15.16:0/3039253775 shutdown_connections 2024-08-17T02:48:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.531+0000 7f7dd9400640 1 -- 172.21.15.16:0/3039253775 wait complete. 2024-08-17T02:48:41.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:48:41.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.698+0000 7feffa800640 1 Processor -- start 2024-08-17T02:48:41.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.699+0000 7feffa800640 1 -- start start 2024-08-17T02:48:41.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.700+0000 7feffa800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feff40cc010 0x7feff40ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.700+0000 7feffa800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7feff40a98e0 0x7feff40afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.700+0000 7feffa800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff40ab950 con 0x7feff40b03d0 2024-08-17T02:48:41.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.700+0000 7feffa800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff40abc30 con 0x7feff40a98e0 2024-08-17T02:48:41.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.700+0000 7feffa800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff40abf10 con 0x7feff40cc010 2024-08-17T02:48:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.700+0000 7feff9400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feff40cc010 0x7feff40ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.700+0000 7feff8a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7feff40a98e0 0x7feff40afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.700+0000 7feff8a00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7feff40a98e0 0x7feff40afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:47838/0 (socket says 172.21.15.16:47838) 2024-08-17T02:48:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.700+0000 7feff8a00640 1 -- 172.21.15.16:0/2702474332 learned_addr learned my addr 172.21.15.16:0/2702474332 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.701+0000 7feff8a00640 1 -- 172.21.15.16:0/2702474332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feff40cc010 msgr2=0x7feff40ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.701+0000 7feff8a00640 1 --2- 172.21.15.16:0/2702474332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feff40cc010 0x7feff40ab2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.701+0000 7feff8a00640 1 -- 172.21.15.16:0/2702474332 >> v1:172.21.15.142:6789/0 conn(0x7feff40b03d0 legacy=0x7feff415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.701+0000 7feff8a00640 1 -- 172.21.15.16:0/2702474332 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff40af970 con 0x7feff40a98e0 2024-08-17T02:48:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.702+0000 7feff8a00640 1 --2- 172.21.15.16:0/2702474332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feff40a98e0 0x7feff40afe90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009910 tx=0x7fefe00091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee222fa8d4490841 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.702+0000 7feff3e00640 1 -- 172.21.15.16:0/2702474332 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe002add0 con 0x7feff40a98e0 2024-08-17T02:48:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.702+0000 7feff3e00640 1 -- 172.21.15.16:0/2702474332 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe0032440 con 0x7feff40a98e0 2024-08-17T02:48:41.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.702+0000 7feff3e00640 1 -- 172.21.15.16:0/2702474332 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0032820 con 0x7feff40a98e0 2024-08-17T02:48:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.703+0000 7feffa800640 1 -- 172.21.15.16:0/2702474332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feff40a98e0 msgr2=0x7feff40afe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.703+0000 7feffa800640 1 --2- 172.21.15.16:0/2702474332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feff40a98e0 0x7feff40afe90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fefe0009910 tx=0x7fefe00091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.703+0000 7feffa800640 1 -- 172.21.15.16:0/2702474332 shutdown_connections 2024-08-17T02:48:41.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.703+0000 7feffa800640 1 --2- 172.21.15.16:0/2702474332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feff40a98e0 0x7feff40afe90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.703+0000 7feffa800640 1 --2- 172.21.15.16:0/2702474332 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feff40cc010 0x7feff40ab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.703+0000 7feffa800640 1 -- 172.21.15.16:0/2702474332 >> 172.21.15.16:0/2702474332 conn(0x7feff4067de0 msgr2=0x7feff40681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.703+0000 7feffa800640 1 -- 172.21.15.16:0/2702474332 shutdown_connections 2024-08-17T02:48:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.703+0000 7feffa800640 1 -- 172.21.15.16:0/2702474332 wait complete. 2024-08-17T02:48:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.704+0000 7feffa800640 1 Processor -- start 2024-08-17T02:48:41.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.705+0000 7feffa800640 1 -- start start 2024-08-17T02:48:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.705+0000 7feffa800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feff40a98e0 0x7feff416c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.705+0000 7feffa800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feff40b03d0 0x7feff4169f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.705+0000 7feffa800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff40cc010 0x7feff415b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feffa800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff4149c70 con 0x7feff40cc010 2024-08-17T02:48:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feffa800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff40c5b30 con 0x7feff40b03d0 2024-08-17T02:48:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feffa800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff416cfb0 con 0x7feff40a98e0 2024-08-17T02:48:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feff9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff40cc010 0x7feff415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feff9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff40cc010 0x7feff415b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41960/0 (socket says 172.21.15.16:41960) 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feff9e00640 1 -- 172.21.15.16:0/1715010457 learned_addr learned my addr 172.21.15.16:0/1715010457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feff9400640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feff40a98e0 0x7feff416c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feff8a00640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feff40b03d0 0x7feff4169f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feff9e00640 1 -- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feff40a98e0 msgr2=0x7feff416c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feff9e00640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feff40a98e0 0x7feff416c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feff9e00640 1 -- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feff40b03d0 msgr2=0x7feff4169f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feff9e00640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feff40b03d0 0x7feff4169f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.706+0000 7feff9e00640 1 -- 172.21.15.16:0/1715010457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe0009450 con 0x7feff40cc010 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.707+0000 7feff9e00640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff40cc010 0x7feff415b380 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fefec00c860 tx=0x7fefec001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.707+0000 7feff0c00640 1 -- 172.21.15.16:0/1715010457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec00ce50 con 0x7feff40cc010 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.707+0000 7feff0c00640 1 -- 172.21.15.16:0/1715010457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec00a850 con 0x7feff40cc010 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.707+0000 7feffa800640 1 -- 172.21.15.16:0/1715010457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff416d290 con 0x7feff40cc010 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.707+0000 7feff0c00640 1 -- 172.21.15.16:0/1715010457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec00ac10 con 0x7feff40cc010 2024-08-17T02:48:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.707+0000 7feffa800640 1 -- 172.21.15.16:0/1715010457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff416d850 con 0x7feff40cc010 2024-08-17T02:48:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.708+0000 7feffa800640 1 -- 172.21.15.16:0/1715010457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff40b1330 con 0x7feff40cc010 2024-08-17T02:48:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.709+0000 7feff0c00640 1 -- 172.21.15.16:0/1715010457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fefec012630 con 0x7feff40cc010 2024-08-17T02:48:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.709+0000 7feff0c00640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fefd4065150 0x7fefd4067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.710+0000 7feff0c00640 1 -- 172.21.15.16:0/1715010457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5837+0+0 (secure 0 0 0) 0x7fefec082660 con 0x7feff40cc010 2024-08-17T02:48:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.710+0000 7feff9400640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fefd4065150 0x7fefd4067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.711+0000 7feff9400640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fefd4065150 0x7fefd4067600 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009c80 tx=0x7fefe80090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.714+0000 7feff0c00640 1 -- 172.21.15.16:0/1715010457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7feff40b1330 con 0x7feff40cc010 2024-08-17T02:48:41.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.896+0000 7feffa800640 1 -- 172.21.15.16:0/1715010457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feff40b1330 con 0x7feff40cc010 2024-08-17T02:48:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.897+0000 7feff0c00640 1 -- 172.21.15.16:0/1715010457 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+15510 (secure 0 0 0) 0x7feff40b1330 con 0x7feff40cc010 2024-08-17T02:48:41.898 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:41.898 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":43,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:48:39.616469+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000521811.meta","create_time":"2024-08-17T02:46:27.102528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000521811"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000521811.data","create_time":"2024-08-17T02:46:27.975177+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000521811"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/219684854":"2024-08-18T02:48:36.607854+0000","172.21.15.16:6833/219684854":"2024-08-18T02:48:36.607854+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:48:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.899+0000 7feffa800640 1 -- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fefd4065150 msgr2=0x7fefd4067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.899+0000 7feffa800640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fefd4065150 0x7fefd4067600 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009c80 tx=0x7fefe80090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.899+0000 7feffa800640 1 -- 172.21.15.16:0/1715010457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff40cc010 msgr2=0x7feff415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.899+0000 7feffa800640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff40cc010 0x7feff415b380 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fefec00c860 tx=0x7fefec001690 comp rx=0 tx=0).stop 2024-08-17T02:48:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.899+0000 7feffa800640 1 -- 172.21.15.16:0/1715010457 shutdown_connections 2024-08-17T02:48:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.899+0000 7feffa800640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fefd4065150 0x7fefd4067600 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.899+0000 7feffa800640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feff40cc010 0x7feff415b380 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.899+0000 7feffa800640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feff40b03d0 0x7feff4169f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.899+0000 7feffa800640 1 --2- 172.21.15.16:0/1715010457 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7feff40a98e0 0x7feff416c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.899+0000 7feffa800640 1 -- 172.21.15.16:0/1715010457 >> 172.21.15.16:0/1715010457 conn(0x7feff4067de0 msgr2=0x7feff4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.899+0000 7feffa800640 1 -- 172.21.15.16:0/1715010457 shutdown_connections 2024-08-17T02:48:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:41.899+0000 7feffa800640 1 -- 172.21.15.16:0/1715010457 wait complete. 2024-08-17T02:48:41.916 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000521811 and related pools 2024-08-17T02:48:41.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:48:42.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.067+0000 7f53d0000640 1 Processor -- start 2024-08-17T02:48:42.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.067+0000 7f53d0000640 1 -- start start 2024-08-17T02:48:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.068+0000 7f53d0000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f53c8155df0 0x7f53c80adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.068+0000 7f53d0000640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c8156670 con 0x7f53c806df00 2024-08-17T02:48:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.068+0000 7f53d0000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c806e5e0 con 0x7f53c8155a20 2024-08-17T02:48:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.068+0000 7f53d0000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c80b0350 con 0x7f53c8155df0 2024-08-17T02:48:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.069+0000 7f53ccc00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f53c806df00 0x7f53c8156b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45270/0 (socket says 172.21.15.16:45270) 2024-08-17T02:48:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.069+0000 7f53ccc00640 1 -- 172.21.15.16:0/774195409 learned_addr learned my addr 172.21.15.16:0/774195409 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.069+0000 7f53ce000640 1 --2- 172.21.15.16:0/774195409 >> v2:172.21.15.142:3300/0 conn(0x7f53c8155df0 0x7f53c80adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.069+0000 7f53c7e00640 1 -- 172.21.15.16:0/774195409 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820867192 0 0) 0x7f53c8156670 con 0x7f53c806df00 2024-08-17T02:48:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.069+0000 7f53c7e00640 1 -- 172.21.15.16:0/774195409 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53b00034e0 con 0x7f53c806df00 2024-08-17T02:48:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.069+0000 7f53ce000640 1 -- 172.21.15.16:0/774195409 >> v1:172.21.15.142:6789/0 conn(0x7f53c8155a20 legacy=0x7f53c806d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.069+0000 7f53ce000640 1 -- 172.21.15.16:0/774195409 >> v1:172.21.15.16:6789/0 conn(0x7f53c806df00 legacy=0x7f53c8156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.069+0000 7f53ce000640 1 -- 172.21.15.16:0/774195409 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c806e8c0 con 0x7f53c8155df0 2024-08-17T02:48:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.071+0000 7f53ce000640 1 --2- 172.21.15.16:0/774195409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c8155df0 0x7f53c80adf10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009e60 tx=0x7f53c0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a324d74502d13d7b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.071+0000 7f53c7e00640 1 -- 172.21.15.16:0/774195409 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0007550 con 0x7f53c8155df0 2024-08-17T02:48:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.071+0000 7f53c7e00640 1 -- 172.21.15.16:0/774195409 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c002d050 con 0x7f53c8155df0 2024-08-17T02:48:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.071+0000 7f53c7e00640 1 -- 172.21.15.16:0/774195409 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c00078e0 con 0x7f53c8155df0 2024-08-17T02:48:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.071+0000 7f53d0000640 1 -- 172.21.15.16:0/774195409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c8155df0 msgr2=0x7f53c80adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.071+0000 7f53d0000640 1 --2- 172.21.15.16:0/774195409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c8155df0 0x7f53c80adf10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009e60 tx=0x7f53c0009780 comp rx=0 tx=0).stop 2024-08-17T02:48:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.072+0000 7f53d0000640 1 -- 172.21.15.16:0/774195409 shutdown_connections 2024-08-17T02:48:42.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.072+0000 7f53d0000640 1 --2- 172.21.15.16:0/774195409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c8155df0 0x7f53c80adf10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.072+0000 7f53d0000640 1 -- 172.21.15.16:0/774195409 >> 172.21.15.16:0/774195409 conn(0x7f53c80cb9b0 msgr2=0x7f53c80cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:42.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.072+0000 7f53d0000640 1 -- 172.21.15.16:0/774195409 shutdown_connections 2024-08-17T02:48:42.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.072+0000 7f53d0000640 1 -- 172.21.15.16:0/774195409 wait complete. 2024-08-17T02:48:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.073+0000 7f53d0000640 1 Processor -- start 2024-08-17T02:48:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.073+0000 7f53d0000640 1 -- start start 2024-08-17T02:48:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.074+0000 7f53d0000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c806df00 0x7f53c80acff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.074+0000 7f53d0000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53c8155a20 0x7f53c80a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.074+0000 7f53d0000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8155df0 0x7f53c814f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.074+0000 7f53d0000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80a8c70 con 0x7f53c8155df0 2024-08-17T02:48:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.074+0000 7f53d0000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80a5b00 con 0x7f53c806df00 2024-08-17T02:48:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.074+0000 7f53d0000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80a86b0 con 0x7f53c8155a20 2024-08-17T02:48:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.074+0000 7f53ce000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8155df0 0x7f53c814f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.074+0000 7f53cd600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c806df00 0x7f53c80acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.074+0000 7f53ccc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53c8155a20 0x7f53c80a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.074+0000 7f53ce000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8155df0 0x7f53c814f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41962/0 (socket says 172.21.15.16:41962) 2024-08-17T02:48:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.075+0000 7f53ce000640 1 -- 172.21.15.16:0/1891688326 learned_addr learned my addr 172.21.15.16:0/1891688326 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.074+0000 7f53cd600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c806df00 0x7f53c80acff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:47878/0 (socket says 172.21.15.16:47878) 2024-08-17T02:48:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.075+0000 7f53ce000640 1 -- 172.21.15.16:0/1891688326 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53c8155a20 msgr2=0x7f53c80a7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.075+0000 7f53ce000640 1 --2- 172.21.15.16:0/1891688326 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53c8155a20 0x7f53c80a7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.075+0000 7f53ce000640 1 -- 172.21.15.16:0/1891688326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c806df00 msgr2=0x7f53c80acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.075+0000 7f53ce000640 1 --2- 172.21.15.16:0/1891688326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c806df00 0x7f53c80acff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.075+0000 7f53ce000640 1 -- 172.21.15.16:0/1891688326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0009a20 con 0x7f53c8155df0 2024-08-17T02:48:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.075+0000 7f53ce000640 1 --2- 172.21.15.16:0/1891688326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8155df0 0x7f53c814f460 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f53c00099f0 tx=0x7f53c0037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.076+0000 7f53c4c00640 1 -- 172.21.15.16:0/1891688326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0031aa0 con 0x7f53c8155df0 2024-08-17T02:48:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.076+0000 7f53c4c00640 1 -- 172.21.15.16:0/1891688326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c0031c00 con 0x7f53c8155df0 2024-08-17T02:48:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.076+0000 7f53c4c00640 1 -- 172.21.15.16:0/1891688326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0045460 con 0x7f53c8155df0 2024-08-17T02:48:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.076+0000 7f53d0000640 1 -- 172.21.15.16:0/1891688326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c80a8990 con 0x7f53c8155df0 2024-08-17T02:48:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.076+0000 7f53d0000640 1 -- 172.21.15.16:0/1891688326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c814f9a0 con 0x7f53c8155df0 2024-08-17T02:48:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.077+0000 7f53d0000640 1 -- 172.21.15.16:0/1891688326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c80a55f0 con 0x7f53c8155df0 2024-08-17T02:48:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.077+0000 7f53c4c00640 1 -- 172.21.15.16:0/1891688326 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f53c003c420 con 0x7f53c8155df0 2024-08-17T02:48:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.078+0000 7f53c4c00640 1 --2- 172.21.15.16:0/1891688326 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f53a8065150 0x7f53a8067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.078+0000 7f53c4c00640 1 -- 172.21.15.16:0/1891688326 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f53c00a3950 con 0x7f53c8155df0 2024-08-17T02:48:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.079+0000 7f53cd600640 1 --2- 172.21.15.16:0/1891688326 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f53a8065150 0x7f53a8067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.079+0000 7f53cd600640 1 --2- 172.21.15.16:0/1891688326 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f53a8065150 0x7f53a8067600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f53bc007fd0 tx=0x7f53bc007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:42.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.082+0000 7f53c4c00640 1 -- 172.21.15.16:0/1891688326 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f53c80a55f0 con 0x7f53c8155df0 2024-08-17T02:48:42.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.300+0000 7f53d0000640 1 -- 172.21.15.16:0/1891688326 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53c80a55f0 con 0x7f53c8155df0 2024-08-17T02:48:42.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.301+0000 7f53c4c00640 1 -- 172.21.15.16:0/1891688326 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4621 (secure 0 0 0) 0x7f53c80a55f0 con 0x7f53c8155df0 2024-08-17T02:48:42.302 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:42.302 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2889208700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2889208700},{"type":"v1","addr":"172.21.15.142:6837","nonce":2889208700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4287834697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4287834697},{"type":"v1","addr":"172.21.15.142:6835","nonce":4287834697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1841153868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1841153868},{"type":"v1","addr":"172.21.15.16:6835","nonce":1841153868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:28.938945+0000","modified":"2024-08-17T02:46:30.943869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6834/3115083427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3115083427},{"type":"v1","addr":"172.21.15.16:6834","nonce":3115083427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000521811","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:48:42.303 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-08-17T02:48:42.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.303+0000 7f53d0000640 1 -- 172.21.15.16:0/1891688326 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f53a8065150 msgr2=0x7f53a8067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.303+0000 7f53d0000640 1 --2- 172.21.15.16:0/1891688326 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f53a8065150 0x7f53a8067600 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f53bc007fd0 tx=0x7f53bc007e30 comp rx=0 tx=0).stop 2024-08-17T02:48:42.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.303+0000 7f53d0000640 1 -- 172.21.15.16:0/1891688326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8155df0 msgr2=0x7f53c814f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.303+0000 7f53d0000640 1 --2- 172.21.15.16:0/1891688326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8155df0 0x7f53c814f460 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f53c00099f0 tx=0x7f53c0037000 comp rx=0 tx=0).stop 2024-08-17T02:48:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.303+0000 7f53d0000640 1 -- 172.21.15.16:0/1891688326 shutdown_connections 2024-08-17T02:48:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.303+0000 7f53d0000640 1 --2- 172.21.15.16:0/1891688326 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f53a8065150 0x7f53a8067600 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.303+0000 7f53d0000640 1 --2- 172.21.15.16:0/1891688326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53c8155df0 0x7f53c814f460 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.303+0000 7f53d0000640 1 --2- 172.21.15.16:0/1891688326 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f53c8155a20 0x7f53c80a7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.303+0000 7f53d0000640 1 --2- 172.21.15.16:0/1891688326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c806df00 0x7f53c80acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.303+0000 7f53d0000640 1 -- 172.21.15.16:0/1891688326 >> 172.21.15.16:0/1891688326 conn(0x7f53c80cb9b0 msgr2=0x7f53c8157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.303+0000 7f53d0000640 1 -- 172.21.15.16:0/1891688326 shutdown_connections 2024-08-17T02:48:42.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.303+0000 7f53d0000640 1 -- 172.21.15.16:0/1891688326 wait complete. 2024-08-17T02:48:42.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:48:42.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.470+0000 7f087f000640 1 Processor -- start 2024-08-17T02:48:42.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.470+0000 7f087f000640 1 -- start start 2024-08-17T02:48:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.471+0000 7f087f000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0878155a20 0x7f08780adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.471+0000 7f087f000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0878155df0 0x7f08780ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.472+0000 7f087f000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0878156670 con 0x7f087806d7f0 2024-08-17T02:48:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.472+0000 7f087f000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08780b0350 con 0x7f0878155a20 2024-08-17T02:48:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.472+0000 7f087f000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087806e5e0 con 0x7f0878155df0 2024-08-17T02:48:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.472+0000 7f0877e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0878155a20 0x7f08780adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.472+0000 7f086f400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0878155df0 0x7f08780ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.472+0000 7f0877e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f0878155a20 0x7f08780adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:47886/0 (socket says 172.21.15.16:47886) 2024-08-17T02:48:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.472+0000 7f0877e00640 1 -- 172.21.15.16:0/2138594420 learned_addr learned my addr 172.21.15.16:0/2138594420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.473+0000 7f086f400640 1 -- 172.21.15.16:0/2138594420 >> v2:172.21.15.142:3300/0 conn(0x7f0878155a20 msgr2=0x7f08780adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:42.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.473+0000 7f086f400640 1 --2- 172.21.15.16:0/2138594420 >> v2:172.21.15.142:3300/0 conn(0x7f0878155a20 0x7f08780adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.473+0000 7f0877400640 1 -- 172.21.15.16:0/2138594420 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807747607 0 0) 0x7f0878156670 con 0x7f087806d7f0 2024-08-17T02:48:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.473+0000 7f086f400640 1 -- 172.21.15.16:0/2138594420 >> v1:172.21.15.142:6789/0 conn(0x7f087806d7f0 legacy=0x7f08780ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.473+0000 7f086f400640 1 -- 172.21.15.16:0/2138594420 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087806e8c0 con 0x7f0878155df0 2024-08-17T02:48:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.474+0000 7f086f400640 1 --2- 172.21.15.16:0/2138594420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0878155df0 0x7f08780ad9b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0860009850 tx=0x7f08600090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92c47691f5795e4b server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.474+0000 7f0877400640 1 -- 172.21.15.16:0/2138594420 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086002ad80 con 0x7f0878155df0 2024-08-17T02:48:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.474+0000 7f0877400640 1 -- 172.21.15.16:0/2138594420 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0860032440 con 0x7f0878155df0 2024-08-17T02:48:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.474+0000 7f0877400640 1 -- 172.21.15.16:0/2138594420 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08600327e0 con 0x7f0878155df0 2024-08-17T02:48:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.474+0000 7f087f000640 1 -- 172.21.15.16:0/2138594420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0878155df0 msgr2=0x7f08780ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.474+0000 7f087f000640 1 --2- 172.21.15.16:0/2138594420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0878155df0 0x7f08780ad9b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0860009850 tx=0x7f08600090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.475+0000 7f087f000640 1 -- 172.21.15.16:0/2138594420 shutdown_connections 2024-08-17T02:48:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.475+0000 7f087f000640 1 --2- 172.21.15.16:0/2138594420 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f0878155df0 0x7f08780ad9b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.475+0000 7f087f000640 1 --2- 172.21.15.16:0/2138594420 >> v2:172.21.15.142:3300/0 conn(0x7f0878155a20 0x7f08780adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.475+0000 7f087f000640 1 -- 172.21.15.16:0/2138594420 >> 172.21.15.16:0/2138594420 conn(0x7f08780cb9b0 msgr2=0x7f08780cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.475+0000 7f087f000640 1 -- 172.21.15.16:0/2138594420 shutdown_connections 2024-08-17T02:48:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.475+0000 7f087f000640 1 -- 172.21.15.16:0/2138594420 wait complete. 2024-08-17T02:48:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.476+0000 7f087f000640 1 Processor -- start 2024-08-17T02:48:42.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.476+0000 7f087f000640 1 -- start start 2024-08-17T02:48:42.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.477+0000 7f087f000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087806d7f0 0x7f087815f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.477+0000 7f087f000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878155a20 0x7f0878144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.477+0000 7f087f000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0878155df0 0x7f087815b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.477+0000 7f087f000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087815f950 con 0x7f0878155a20 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.477+0000 7f087f000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08780aa340 con 0x7f0878155df0 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.477+0000 7f087f000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08780a9a10 con 0x7f087806d7f0 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.477+0000 7f086f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878155a20 0x7f0878144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.477+0000 7f086f400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878155a20 0x7f0878144490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41964/0 (socket says 172.21.15.16:41964) 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.477+0000 7f086f400640 1 -- 172.21.15.16:0/1029424119 learned_addr learned my addr 172.21.15.16:0/1029424119 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.477+0000 7f0877e00640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087806d7f0 0x7f087815f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.477+0000 7f087d000640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0878155df0 0x7f087815b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.478+0000 7f086f400640 1 -- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087806d7f0 msgr2=0x7f087815f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.478+0000 7f086f400640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087806d7f0 0x7f087815f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.478+0000 7f086f400640 1 -- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0878155df0 msgr2=0x7f087815b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.478+0000 7f086f400640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0878155df0 0x7f087815b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.478+0000 7f086f400640 1 -- 172.21.15.16:0/1029424119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0860009390 con 0x7f0878155a20 2024-08-17T02:48:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.478+0000 7f086f400640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878155a20 0x7f0878144490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0860009850 tx=0x7f0860032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.478+0000 7f086fe00640 1 -- 172.21.15.16:0/1029424119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860031de0 con 0x7f0878155a20 2024-08-17T02:48:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.478+0000 7f086fe00640 1 -- 172.21.15.16:0/1029424119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086003e450 con 0x7f0878155a20 2024-08-17T02:48:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.478+0000 7f087f000640 1 -- 172.21.15.16:0/1029424119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08780a9cf0 con 0x7f0878155a20 2024-08-17T02:48:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.478+0000 7f086fe00640 1 -- 172.21.15.16:0/1029424119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086003e850 con 0x7f0878155a20 2024-08-17T02:48:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.478+0000 7f087f000640 1 -- 172.21.15.16:0/1029424119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087815f660 con 0x7f0878155a20 2024-08-17T02:48:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.480+0000 7f087f000640 1 -- 172.21.15.16:0/1029424119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0878064db0 con 0x7f0878155a20 2024-08-17T02:48:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.480+0000 7f086fe00640 1 -- 172.21.15.16:0/1029424119 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f086003c400 con 0x7f0878155a20 2024-08-17T02:48:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.481+0000 7f086fe00640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0854065150 0x7f0854067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.481+0000 7f086fe00640 1 -- 172.21.15.16:0/1029424119 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f08600a2f90 con 0x7f0878155a20 2024-08-17T02:48:42.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.482+0000 7f0877e00640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0854065150 0x7f0854067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.482+0000 7f0877e00640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0854065150 0x7f0854067600 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f086800bea0 tx=0x7f086800b460 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:42.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.485+0000 7f086fe00640 1 -- 172.21.15.16:0/1029424119 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f0860070fe0 con 0x7f0878155a20 2024-08-17T02:48:42.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.705+0000 7f087f000640 1 -- 172.21.15.16:0/1029424119 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f087806d470 con 0x7f0878155a20 2024-08-17T02:48:42.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.705+0000 7f086fe00640 1 -- 172.21.15.16:0/1029424119 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4621 (secure 0 0 0) 0x7f0860074600 con 0x7f0878155a20 2024-08-17T02:48:42.707 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:42.707 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2889208700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2889208700},{"type":"v1","addr":"172.21.15.142:6837","nonce":2889208700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4287834697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4287834697},{"type":"v1","addr":"172.21.15.142:6835","nonce":4287834697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1841153868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1841153868},{"type":"v1","addr":"172.21.15.16:6835","nonce":1841153868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-17T02:46:28.938945+0000","modified":"2024-08-17T02:46:30.943869+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":9,"addr":"172.21.15.16:6834/3115083427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3115083427},{"type":"v1","addr":"172.21.15.16:6834","nonce":3115083427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000521811","balancer":"","standby_count_wanted":1},"id":3}]} 2024-08-17T02:48:42.707 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 24 2024-08-17T02:48:42.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.708+0000 7f087f000640 1 -- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0854065150 msgr2=0x7f0854067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.708+0000 7f087f000640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0854065150 0x7f0854067600 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f086800bea0 tx=0x7f086800b460 comp rx=0 tx=0).stop 2024-08-17T02:48:42.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.708+0000 7f087f000640 1 -- 172.21.15.16:0/1029424119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878155a20 msgr2=0x7f0878144490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.708+0000 7f087f000640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878155a20 0x7f0878144490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0860009850 tx=0x7f0860032000 comp rx=0 tx=0).stop 2024-08-17T02:48:42.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.708+0000 7f087f000640 1 -- 172.21.15.16:0/1029424119 shutdown_connections 2024-08-17T02:48:42.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.708+0000 7f087f000640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f0854065150 0x7f0854067600 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.708+0000 7f087f000640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0878155df0 0x7f087815b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.708+0000 7f087f000640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0878155a20 0x7f0878144490 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.709 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.708+0000 7f087f000640 1 --2- 172.21.15.16:0/1029424119 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f087806d7f0 0x7f087815f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.708+0000 7f087f000640 1 -- 172.21.15.16:0/1029424119 >> 172.21.15.16:0/1029424119 conn(0x7f08780cb9b0 msgr2=0x7f0878153620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:42.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.708+0000 7f087f000640 1 -- 172.21.15.16:0/1029424119 shutdown_connections 2024-08-17T02:48:42.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.708+0000 7f087f000640 1 -- 172.21.15.16:0/1029424119 wait complete. 2024-08-17T02:48:42.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:48:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.873+0000 7f28f5000640 1 Processor -- start 2024-08-17T02:48:42.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.873+0000 7f28f5000640 1 -- start start 2024-08-17T02:48:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28f5000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f28f00b0270 0x7f28f01563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28f5000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28f0158b30 0x7f28f00ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28f5000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f0157690 con 0x7f28f0155900 2024-08-17T02:48:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28f5000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f0156a80 con 0x7f28f0158b30 2024-08-17T02:48:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28f5000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f0156d60 con 0x7f28f00b0270 2024-08-17T02:48:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28efe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28f0158b30 0x7f28f00ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28efe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28f0158b30 0x7f28f00ae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41978/0 (socket says 172.21.15.16:41978) 2024-08-17T02:48:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28efe00640 1 -- 172.21.15.16:0/3013949864 learned_addr learned my addr 172.21.15.16:0/3013949864 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28e7e00640 1 --2- 172.21.15.16:0/3013949864 >> v2:172.21.15.142:3300/0 conn(0x7f28f00b0270 0x7f28f01563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28efe00640 1 -- 172.21.15.16:0/3013949864 >> v2:172.21.15.142:3300/0 conn(0x7f28f00b0270 msgr2=0x7f28f01563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28efe00640 1 --2- 172.21.15.16:0/3013949864 >> v2:172.21.15.142:3300/0 conn(0x7f28f00b0270 0x7f28f01563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28efe00640 1 -- 172.21.15.16:0/3013949864 >> v1:172.21.15.142:6789/0 conn(0x7f28f0155900 legacy=0x7f28f0155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.875+0000 7f28efe00640 1 -- 172.21.15.16:0/3013949864 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f0157040 con 0x7f28f0158b30 2024-08-17T02:48:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.876+0000 7f28efe00640 1 --2- 172.21.15.16:0/3013949864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0158b30 0x7f28f00ae6b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f28f006f430 tx=0x7f28e8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1aae28d2d8d379 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.877+0000 7f28eea00640 1 -- 172.21.15.16:0/3013949864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8032400 con 0x7f28f0158b30 2024-08-17T02:48:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.877+0000 7f28eea00640 1 -- 172.21.15.16:0/3013949864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e802e050 con 0x7f28f0158b30 2024-08-17T02:48:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.877+0000 7f28eea00640 1 -- 172.21.15.16:0/3013949864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80327f0 con 0x7f28f0158b30 2024-08-17T02:48:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.877+0000 7f28f5000640 1 -- 172.21.15.16:0/3013949864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0158b30 msgr2=0x7f28f00ae6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.877+0000 7f28f5000640 1 --2- 172.21.15.16:0/3013949864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0158b30 0x7f28f00ae6b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f28f006f430 tx=0x7f28e8009840 comp rx=0 tx=0).stop 2024-08-17T02:48:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.877+0000 7f28f5000640 1 -- 172.21.15.16:0/3013949864 shutdown_connections 2024-08-17T02:48:42.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.877+0000 7f28f5000640 1 --2- 172.21.15.16:0/3013949864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0158b30 0x7f28f00ae6b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.877+0000 7f28f5000640 1 --2- 172.21.15.16:0/3013949864 >> v2:172.21.15.142:3300/0 conn(0x7f28f00b0270 0x7f28f01563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.877+0000 7f28f5000640 1 -- 172.21.15.16:0/3013949864 >> 172.21.15.16:0/3013949864 conn(0x7f28f0067750 msgr2=0x7f28f00cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.877+0000 7f28f5000640 1 -- 172.21.15.16:0/3013949864 shutdown_connections 2024-08-17T02:48:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.877+0000 7f28f5000640 1 -- 172.21.15.16:0/3013949864 wait complete. 2024-08-17T02:48:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.879+0000 7f28f5000640 1 Processor -- start 2024-08-17T02:48:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.879+0000 7f28f5000640 1 -- start start 2024-08-17T02:48:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.880+0000 7f28f5000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f00b0270 0x7f28f00a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.880+0000 7f28f5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f0155900 0x7f28f016a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.880+0000 7f28f5000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0158b30 0x7f28f0153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.880+0000 7f28f5000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00a6510 con 0x7f28f0158b30 2024-08-17T02:48:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.880+0000 7f28f5000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f016c110 con 0x7f28f0155900 2024-08-17T02:48:42.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.880+0000 7f28f5000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f016bc80 con 0x7f28f00b0270 2024-08-17T02:48:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.880+0000 7f28efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0158b30 0x7f28f0153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.880+0000 7f28efe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0158b30 0x7f28f0153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41992/0 (socket says 172.21.15.16:41992) 2024-08-17T02:48:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.880+0000 7f28efe00640 1 -- 172.21.15.16:0/467694756 learned_addr learned my addr 172.21.15.16:0/467694756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.880+0000 7f28e7e00640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f0155900 0x7f28f016a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.880+0000 7f28ef400640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f00b0270 0x7f28f00a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.881+0000 7f28efe00640 1 -- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f00b0270 msgr2=0x7f28f00a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.881+0000 7f28efe00640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f00b0270 0x7f28f00a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.881+0000 7f28efe00640 1 -- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f0155900 msgr2=0x7f28f016a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.881+0000 7f28efe00640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f0155900 0x7f28f016a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.881+0000 7f28efe00640 1 -- 172.21.15.16:0/467694756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e8009ae0 con 0x7f28f0158b30 2024-08-17T02:48:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.881+0000 7f28ef400640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f00b0270 0x7f28f00a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:42.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.881+0000 7f28efe00640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0158b30 0x7f28f0153d50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009f50 tx=0x7f28e8037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.881+0000 7f28e6a00640 1 -- 172.21.15.16:0/467694756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80077e0 con 0x7f28f0158b30 2024-08-17T02:48:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.881+0000 7f28e6a00640 1 -- 172.21.15.16:0/467694756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e8007960 con 0x7f28f0158b30 2024-08-17T02:48:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.881+0000 7f28e6a00640 1 -- 172.21.15.16:0/467694756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8007d20 con 0x7f28f0158b30 2024-08-17T02:48:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.881+0000 7f28f5000640 1 -- 172.21.15.16:0/467694756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f016ae10 con 0x7f28f0158b30 2024-08-17T02:48:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.881+0000 7f28f5000640 1 -- 172.21.15.16:0/467694756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f016b350 con 0x7f28f0158b30 2024-08-17T02:48:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.883+0000 7f28f5000640 1 -- 172.21.15.16:0/467694756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f0071050 con 0x7f28f0158b30 2024-08-17T02:48:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.883+0000 7f28e6a00640 1 -- 172.21.15.16:0/467694756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f28e8003040 con 0x7f28f0158b30 2024-08-17T02:48:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.884+0000 7f28e6a00640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f28cc065150 0x7f28cc067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.885+0000 7f28e6a00640 1 -- 172.21.15.16:0/467694756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f28e80a2900 con 0x7f28f0158b30 2024-08-17T02:48:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.885+0000 7f28ef400640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f28cc065150 0x7f28cc067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:42.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.886+0000 7f28ef400640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f28cc065150 0x7f28cc067600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f28e0001820 tx=0x7f28e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:42.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:42.889+0000 7f28e6a00640 1 -- 172.21.15.16:0/467694756 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f28e80709f0 con 0x7f28f0158b30 2024-08-17T02:48:43.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.070+0000 7f28f5000640 1 -- 172.21.15.16:0/467694756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28f016bdc0 con 0x7f28f0158b30 2024-08-17T02:48:43.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.071+0000 7f28e6a00640 1 -- 172.21.15.16:0/467694756 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+15510 (secure 0 0 0) 0x7f28e8074010 con 0x7f28f0158b30 2024-08-17T02:48:43.072 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:43.073 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":43,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:48:39.616469+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs.volume_0000000000521811.meta","create_time":"2024-08-17T02:46:27.102528+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000521811"}}},{"pool":7,"pool_name":"cephfs.volume_0000000000521811.data","create_time":"2024-08-17T02:46:27.975177+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000521811"}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6835/219684854":"2024-08-18T02:48:36.607854+0000","172.21.15.16:6833/219684854":"2024-08-18T02:48:36.607854+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:48:43.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.073+0000 7f28f5000640 1 -- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f28cc065150 msgr2=0x7f28cc067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:43.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.073+0000 7f28f5000640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f28cc065150 0x7f28cc067600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f28e0001820 tx=0x7f28e0009040 comp rx=0 tx=0).stop 2024-08-17T02:48:43.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.073+0000 7f28f5000640 1 -- 172.21.15.16:0/467694756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0158b30 msgr2=0x7f28f0153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:43.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.073+0000 7f28f5000640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0158b30 0x7f28f0153d50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009f50 tx=0x7f28e8037000 comp rx=0 tx=0).stop 2024-08-17T02:48:43.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.073+0000 7f28f5000640 1 -- 172.21.15.16:0/467694756 shutdown_connections 2024-08-17T02:48:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.073+0000 7f28f5000640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f28cc065150 0x7f28cc067600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.073+0000 7f28f5000640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f0158b30 0x7f28f0153d50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.073+0000 7f28f5000640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28f0155900 0x7f28f016a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.073+0000 7f28f5000640 1 --2- 172.21.15.16:0/467694756 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f28f00b0270 0x7f28f00a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.073+0000 7f28f5000640 1 -- 172.21.15.16:0/467694756 >> 172.21.15.16:0/467694756 conn(0x7f28f0067750 msgr2=0x7f28f0068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.073+0000 7f28f5000640 1 -- 172.21.15.16:0/467694756 shutdown_connections 2024-08-17T02:48:43.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.073+0000 7f28f5000640 1 -- 172.21.15.16:0/467694756 wait complete. 2024-08-17T02:48:43.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000521811 2024-08-17T02:48:43.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.241+0000 7f873e400640 1 Processor -- start 2024-08-17T02:48:43.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.241+0000 7f873e400640 1 -- start start 2024-08-17T02:48:43.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.242+0000 7f873e400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87381401b0 0x7f87380caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:43.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.242+0000 7f873e400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87380b1440 0x7f87380c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:43.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.242+0000 7f873e400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87380caa80 con 0x7f8738140580 2024-08-17T02:48:43.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.242+0000 7f873e400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87380cb450 con 0x7f87380b1440 2024-08-17T02:48:43.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.242+0000 7f873e400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87380c9e70 con 0x7f87381401b0 2024-08-17T02:48:43.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.242+0000 7f8737e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87380b1440 0x7f87380c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:43.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.242+0000 7f8737e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87380b1440 0x7f87380c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41996/0 (socket says 172.21.15.16:41996) 2024-08-17T02:48:43.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.242+0000 7f8737e00640 1 -- 172.21.15.16:0/2466860592 learned_addr learned my addr 172.21.15.16:0/2466860592 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:43.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.243+0000 7f8737400640 1 --2- 172.21.15.16:0/2466860592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87381401b0 0x7f87380caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:43.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.243+0000 7f8737e00640 1 -- 172.21.15.16:0/2466860592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87381401b0 msgr2=0x7f87380caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:43.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.243+0000 7f8737e00640 1 --2- 172.21.15.16:0/2466860592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87381401b0 0x7f87380caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.243+0000 7f8737e00640 1 -- 172.21.15.16:0/2466860592 >> v1:172.21.15.142:6789/0 conn(0x7f8738140580 legacy=0x7f87380b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:43.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.243+0000 7f8737e00640 1 -- 172.21.15.16:0/2466860592 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87380ca150 con 0x7f87380b1440 2024-08-17T02:48:43.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.244+0000 7f8737e00640 1 --2- 172.21.15.16:0/2466860592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87380b1440 0x7f87380c9040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8730009f20 tx=0x7f8730009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ddd7a7797e75b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:43.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.244+0000 7f8736000640 1 -- 172.21.15.16:0/2466860592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730031400 con 0x7f87380b1440 2024-08-17T02:48:43.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.244+0000 7f8736000640 1 -- 172.21.15.16:0/2466860592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873002d050 con 0x7f87380b1440 2024-08-17T02:48:43.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.244+0000 7f8736000640 1 -- 172.21.15.16:0/2466860592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87300317d0 con 0x7f87380b1440 2024-08-17T02:48:43.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.245+0000 7f873e400640 1 -- 172.21.15.16:0/2466860592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87380b1440 msgr2=0x7f87380c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:43.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.245+0000 7f873e400640 1 --2- 172.21.15.16:0/2466860592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87380b1440 0x7f87380c9040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8730009f20 tx=0x7f8730009840 comp rx=0 tx=0).stop 2024-08-17T02:48:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.245+0000 7f873e400640 1 -- 172.21.15.16:0/2466860592 shutdown_connections 2024-08-17T02:48:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.245+0000 7f873e400640 1 --2- 172.21.15.16:0/2466860592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87380b1440 0x7f87380c9040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.245+0000 7f873e400640 1 --2- 172.21.15.16:0/2466860592 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87381401b0 0x7f87380caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.245+0000 7f873e400640 1 -- 172.21.15.16:0/2466860592 >> 172.21.15.16:0/2466860592 conn(0x7f87380cb9b0 msgr2=0x7f87380cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.245+0000 7f873e400640 1 -- 172.21.15.16:0/2466860592 shutdown_connections 2024-08-17T02:48:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.245+0000 7f873e400640 1 -- 172.21.15.16:0/2466860592 wait complete. 2024-08-17T02:48:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.246+0000 7f873e400640 1 Processor -- start 2024-08-17T02:48:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.246+0000 7f873e400640 1 -- start start 2024-08-17T02:48:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.247+0000 7f873e400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87380b1440 0x7f87380c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.247+0000 7f873e400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87381401b0 0x7f87380aa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.247+0000 7f873e400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738140580 0x7f87380aa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.247+0000 7f873e400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8738169000 con 0x7f8738140580 2024-08-17T02:48:43.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.247+0000 7f873e400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87380ad050 con 0x7f87380b1440 2024-08-17T02:48:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.247+0000 7f873e400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87380ad330 con 0x7f87381401b0 2024-08-17T02:48:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8737e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738140580 0x7f87380aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8737e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738140580 0x7f87380aa730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42008/0 (socket says 172.21.15.16:42008) 2024-08-17T02:48:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8737e00640 1 -- 172.21.15.16:0/1825180493 learned_addr learned my addr 172.21.15.16:0/1825180493 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8737400640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87380b1440 0x7f87380c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8737e00640 1 -- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87381401b0 msgr2=0x7f87380aa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8736a00640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87381401b0 0x7f87380aa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8737e00640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87381401b0 0x7f87380aa1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8737e00640 1 -- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87380b1440 msgr2=0x7f87380c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8737e00640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87380b1440 0x7f87380c7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8737e00640 1 -- 172.21.15.16:0/1825180493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8730009ae0 con 0x7f8738140580 2024-08-17T02:48:43.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8737400640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87380b1440 0x7f87380c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8736a00640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87381401b0 0x7f87380aa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.248+0000 7f8737e00640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738140580 0x7f87380aa730 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8730009f80 tx=0x7f8730036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.249+0000 7f872ea00640 1 -- 172.21.15.16:0/1825180493 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730008e50 con 0x7f8738140580 2024-08-17T02:48:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.249+0000 7f872ea00640 1 -- 172.21.15.16:0/1825180493 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873003b410 con 0x7f8738140580 2024-08-17T02:48:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.249+0000 7f873e400640 1 -- 172.21.15.16:0/1825180493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87380ad610 con 0x7f8738140580 2024-08-17T02:48:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.249+0000 7f872ea00640 1 -- 172.21.15.16:0/1825180493 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873003b7d0 con 0x7f8738140580 2024-08-17T02:48:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.249+0000 7f873e400640 1 -- 172.21.15.16:0/1825180493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87380aeb50 con 0x7f8738140580 2024-08-17T02:48:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.250+0000 7f873e400640 1 -- 172.21.15.16:0/1825180493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8738064db0 con 0x7f8738140580 2024-08-17T02:48:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.251+0000 7f872ea00640 1 -- 172.21.15.16:0/1825180493 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f8730038050 con 0x7f8738140580 2024-08-17T02:48:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.251+0000 7f872ea00640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8718065150 0x7f8718067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.252+0000 7f872ea00640 1 -- 172.21.15.16:0/1825180493 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5837+0+0 (secure 0 0 0) 0x7f87300a3100 con 0x7f8738140580 2024-08-17T02:48:43.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.252+0000 7f8737400640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8718065150 0x7f8718067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:43.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.253+0000 7f8737400640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8718065150 0x7f8718067600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8728006e70 tx=0x7f8728008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:43.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.256+0000 7f872ea00640 1 -- 172.21.15.16:0/1825180493 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f87300710a0 con 0x7f8738140580 2024-08-17T02:48:43.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.479+0000 7f873e400640 1 -- 172.21.15.16:0/1825180493 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000521811"} v 0) v1 -- 0x7f87380adce0 con 0x7f8738140580 2024-08-17T02:48:43.481 INFO:tasks.ceph.mon.a.smithi016.stderr:2024-08-17T02:48:43.480+0000 7f23cec00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-17T02:48:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.484+0000 7f872ea00640 1 -- 172.21.15.16:0/1825180493 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000521811"}]=0 volume_0000000000521811 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 195+0+0 (secure 0 0 0) 0x7f87300746c0 con 0x7f8738140580 2024-08-17T02:48:43.485 INFO:teuthology.orchestra.run.smithi016.stderr:volume_0000000000521811 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-17T02:48:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.487+0000 7f873e400640 1 -- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8718065150 msgr2=0x7f8718067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.487+0000 7f873e400640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8718065150 0x7f8718067600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8728006e70 tx=0x7f8728008040 comp rx=0 tx=0).stop 2024-08-17T02:48:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.487+0000 7f873e400640 1 -- 172.21.15.16:0/1825180493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738140580 msgr2=0x7f87380aa730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.487+0000 7f873e400640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738140580 0x7f87380aa730 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8730009f80 tx=0x7f8730036000 comp rx=0 tx=0).stop 2024-08-17T02:48:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.487+0000 7f873e400640 1 -- 172.21.15.16:0/1825180493 shutdown_connections 2024-08-17T02:48:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.487+0000 7f873e400640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8718065150 0x7f8718067600 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.487+0000 7f873e400640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738140580 0x7f87380aa730 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.487+0000 7f873e400640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f87381401b0 0x7f87380aa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.487+0000 7f873e400640 1 --2- 172.21.15.16:0/1825180493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87380b1440 0x7f87380c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.487+0000 7f873e400640 1 -- 172.21.15.16:0/1825180493 >> 172.21.15.16:0/1825180493 conn(0x7f87380cb9b0 msgr2=0x7f87380c09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.487+0000 7f873e400640 1 -- 172.21.15.16:0/1825180493 shutdown_connections 2024-08-17T02:48:43.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.487+0000 7f873e400640 1 -- 172.21.15.16:0/1825180493 wait complete. 2024-08-17T02:48:43.492 INFO:tasks.ceph.mds.c.smithi016.stderr: -3155> 2024-08-17T02:46:33.721+0000 7f73eb600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-17T02:48:43.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000521811 --yes-i-really-mean-it 2024-08-17T02:48:43.535 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2024-08-17T02:48:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.660+0000 7f01a3200640 1 Processor -- start 2024-08-17T02:48:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.660+0000 7f01a3200640 1 -- start start 2024-08-17T02:48:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.661+0000 7f01a3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f019c155980 0x7f019c156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.661+0000 7f01a3200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f019c0af580 0x7f019c0afa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.661+0000 7f01a3200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c158400 con 0x7f019c0a54d0 2024-08-17T02:48:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.661+0000 7f01a3200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019c0aff40 con 0x7f019c0af580 2024-08-17T02:48:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.661+0000 7f01a3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019c1577f0 con 0x7f019c155980 2024-08-17T02:48:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.662+0000 7f01a1200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f019c0af580 0x7f019c0afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.662+0000 7f019b400640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f019c0a54d0 0x7f019c155d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45280/0 (socket says 172.21.15.16:45280) 2024-08-17T02:48:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.662+0000 7f019b400640 1 -- 172.21.15.16:0/2837691 learned_addr learned my addr 172.21.15.16:0/2837691 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.662+0000 7f01a1200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f019c0af580 0x7f019c0afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42034/0 (socket says 172.21.15.16:42034) 2024-08-17T02:48:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.662+0000 7f019be00640 1 --2- 172.21.15.16:0/2837691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f019c155980 0x7f019c156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.662+0000 7f01a1200640 1 -- 172.21.15.16:0/2837691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f019c155980 msgr2=0x7f019c156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.662+0000 7f01a1200640 1 --2- 172.21.15.16:0/2837691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f019c155980 0x7f019c156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.662+0000 7f01a1200640 1 -- 172.21.15.16:0/2837691 >> v1:172.21.15.16:6789/0 conn(0x7f019c0a54d0 legacy=0x7f019c155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:43.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.662+0000 7f01a1200640 1 -- 172.21.15.16:0/2837691 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c157ad0 con 0x7f019c0af580 2024-08-17T02:48:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.663+0000 7f01a1200640 1 --2- 172.21.15.16:0/2837691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c0af580 0x7f019c0afa00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0194009e60 tx=0x7f0194009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd146ab71a858eaf server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.663+0000 7f019aa00640 1 -- 172.21.15.16:0/2837691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194007550 con 0x7f019c0af580 2024-08-17T02:48:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.663+0000 7f019aa00640 1 -- 172.21.15.16:0/2837691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019402d050 con 0x7f019c0af580 2024-08-17T02:48:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.663+0000 7f019aa00640 1 -- 172.21.15.16:0/2837691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194007920 con 0x7f019c0af580 2024-08-17T02:48:43.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.664+0000 7f01a3200640 1 -- 172.21.15.16:0/2837691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c0af580 msgr2=0x7f019c0afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.664+0000 7f01a3200640 1 --2- 172.21.15.16:0/2837691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c0af580 0x7f019c0afa00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0194009e60 tx=0x7f0194009780 comp rx=0 tx=0).stop 2024-08-17T02:48:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.664+0000 7f01a3200640 1 -- 172.21.15.16:0/2837691 shutdown_connections 2024-08-17T02:48:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.664+0000 7f01a3200640 1 --2- 172.21.15.16:0/2837691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c0af580 0x7f019c0afa00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.664+0000 7f01a3200640 1 --2- 172.21.15.16:0/2837691 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f019c155980 0x7f019c156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.664+0000 7f01a3200640 1 -- 172.21.15.16:0/2837691 >> 172.21.15.16:0/2837691 conn(0x7f019c067750 msgr2=0x7f019c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.664+0000 7f01a3200640 1 -- 172.21.15.16:0/2837691 shutdown_connections 2024-08-17T02:48:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.664+0000 7f01a3200640 1 -- 172.21.15.16:0/2837691 wait complete. 2024-08-17T02:48:43.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.665+0000 7f01a3200640 1 Processor -- start 2024-08-17T02:48:43.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.666+0000 7f01a3200640 1 -- start start 2024-08-17T02:48:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a3200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f019c0a54d0 0x7f019c0acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f019c0af580 0x7f019c14a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a3200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c155980 0x7f019c0a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:43.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a3200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0adf40 con 0x7f019c155980 2024-08-17T02:48:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c1690c0 con 0x7f019c0af580 2024-08-17T02:48:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a3200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0ad6a0 con 0x7f019c0a54d0 2024-08-17T02:48:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c155980 0x7f019c0a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a1200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c155980 0x7f019c0a9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42038/0 (socket says 172.21.15.16:42038) 2024-08-17T02:48:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a1200640 1 -- 172.21.15.16:0/2147041786 learned_addr learned my addr 172.21.15.16:0/2147041786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f019be00640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f019c0a54d0 0x7f019c0acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f019b400640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f019c0af580 0x7f019c14a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a1200640 1 -- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f019c0a54d0 msgr2=0x7f019c0acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a1200640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f019c0a54d0 0x7f019c0acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a1200640 1 -- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f019c0af580 msgr2=0x7f019c14a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a1200640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f019c0af580 0x7f019c14a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:43.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.667+0000 7f01a1200640 1 -- 172.21.15.16:0/2147041786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194009a20 con 0x7f019c155980 2024-08-17T02:48:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.668+0000 7f01a1200640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c155980 0x7f019c0a9cb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0194009ec0 tx=0x7f0194037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.668+0000 7f0193400640 1 -- 172.21.15.16:0/2147041786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194031890 con 0x7f019c155980 2024-08-17T02:48:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.668+0000 7f0193400640 1 -- 172.21.15.16:0/2147041786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0194040070 con 0x7f019c155980 2024-08-17T02:48:43.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.668+0000 7f0193400640 1 -- 172.21.15.16:0/2147041786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194031c20 con 0x7f019c155980 2024-08-17T02:48:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.668+0000 7f01a3200640 1 -- 172.21.15.16:0/2147041786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019c0ad980 con 0x7f019c155980 2024-08-17T02:48:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.668+0000 7f01a3200640 1 -- 172.21.15.16:0/2147041786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019c14ab70 con 0x7f019c155980 2024-08-17T02:48:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.669+0000 7f01a3200640 1 -- 172.21.15.16:0/2147041786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019c168e90 con 0x7f019c155980 2024-08-17T02:48:43.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.670+0000 7f0193400640 1 -- 172.21.15.16:0/2147041786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f0194042030 con 0x7f019c155980 2024-08-17T02:48:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.671+0000 7f0193400640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f017c065150 0x7f017c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.671+0000 7f0193400640 1 -- 172.21.15.16:0/2147041786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5923+0+0 (secure 0 0 0) 0x7f01940a3d10 con 0x7f019c155980 2024-08-17T02:48:43.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.671+0000 7f019be00640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f017c065150 0x7f017c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:43.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.672+0000 7f019be00640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f017c065150 0x7f017c067600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f018c009d20 tx=0x7f018c0090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:43.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.675+0000 7f0193400640 1 -- 172.21.15.16:0/2147041786 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f019c168e90 con 0x7f019c155980 2024-08-17T02:48:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:43.894+0000 7f01a3200640 1 -- 172.21.15.16:0/2147041786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000521811", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f019c168e90 con 0x7f019c155980 2024-08-17T02:48:44.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.486+0000 7f0193400640 1 -- 172.21.15.16:0/2147041786 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000521811", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 121+0+0 (secure 0 0 0) 0x7f019c168e90 con 0x7f019c155980 2024-08-17T02:48:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.488+0000 7f01a3200640 1 -- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f017c065150 msgr2=0x7f017c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.488+0000 7f01a3200640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f017c065150 0x7f017c067600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f018c009d20 tx=0x7f018c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.488+0000 7f01a3200640 1 -- 172.21.15.16:0/2147041786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c155980 msgr2=0x7f019c0a9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.488+0000 7f01a3200640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c155980 0x7f019c0a9cb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0194009ec0 tx=0x7f0194037000 comp rx=0 tx=0).stop 2024-08-17T02:48:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.488+0000 7f01a3200640 1 -- 172.21.15.16:0/2147041786 shutdown_connections 2024-08-17T02:48:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.488+0000 7f01a3200640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f017c065150 0x7f017c067600 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.488+0000 7f01a3200640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c155980 0x7f019c0a9cb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.488+0000 7f01a3200640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f019c0af580 0x7f019c14a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.488+0000 7f01a3200640 1 --2- 172.21.15.16:0/2147041786 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f019c0a54d0 0x7f019c0acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.488+0000 7f01a3200640 1 -- 172.21.15.16:0/2147041786 >> 172.21.15.16:0/2147041786 conn(0x7f019c067750 msgr2=0x7f019c06e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:44.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.488+0000 7f01a3200640 1 -- 172.21.15.16:0/2147041786 shutdown_connections 2024-08-17T02:48:44.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.489+0000 7f01a3200640 1 -- 172.21.15.16:0/2147041786 wait complete. 2024-08-17T02:48:44.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000521811.meta cephfs.volume_0000000000521811.meta --yes-i-really-really-mean-it 2024-08-17T02:48:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.704+0000 7fcf7a400640 1 Processor -- start 2024-08-17T02:48:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.704+0000 7fcf7a400640 1 -- start start 2024-08-17T02:48:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.705+0000 7fcf7a400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf74158f30 0x7fcf740a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.705+0000 7fcf7a400640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf740aa960 con 0x7fcf740a9e20 2024-08-17T02:48:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.706+0000 7fcf7a400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf740ab6a0 con 0x7fcf74158b60 2024-08-17T02:48:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.706+0000 7fcf7a400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf740ac240 con 0x7fcf74158f30 2024-08-17T02:48:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.706+0000 7fcf73e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf74158f30 0x7fcf740a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.706+0000 7fcf73e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf74158f30 0x7fcf740a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39544/0 (socket says 172.21.15.16:39544) 2024-08-17T02:48:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.706+0000 7fcf73e00640 1 -- 172.21.15.16:0/184977546 learned_addr learned my addr 172.21.15.16:0/184977546 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.706+0000 7fcf79a00640 1 --1- 172.21.15.16:0/184977546 >> v1:172.21.15.16:6789/0 conn(0x7fcf740a9e20 0x7fcf740ac500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34988/0 (socket says 172.21.15.16:34988) 2024-08-17T02:48:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.706+0000 7fcf73e00640 1 -- 172.21.15.16:0/184977546 >> v1:172.21.15.142:6789/0 conn(0x7fcf74158b60 legacy=0x7fcf740abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:44.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.706+0000 7fcf73e00640 1 -- 172.21.15.16:0/184977546 >> v1:172.21.15.16:6789/0 conn(0x7fcf740a9e20 legacy=0x7fcf740ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.706+0000 7fcf73e00640 1 -- 172.21.15.16:0/184977546 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf740aa3a0 con 0x7fcf74158f30 2024-08-17T02:48:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.706+0000 7fcf73400640 1 -- 172.21.15.16:0/184977546 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713649942 0 0) 0x7fcf740aa960 con 0x7fcf740a9e20 2024-08-17T02:48:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.707+0000 7fcf73e00640 1 --2- 172.21.15.16:0/184977546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf74158f30 0x7fcf740a98e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009910 tx=0x7fcf600091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8604d3f1b5dc2d52 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.708+0000 7fcf73400640 1 -- 172.21.15.16:0/184977546 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6002ae60 con 0x7fcf74158f30 2024-08-17T02:48:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.708+0000 7fcf73400640 1 -- 172.21.15.16:0/184977546 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf60032440 con 0x7fcf74158f30 2024-08-17T02:48:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.708+0000 7fcf73400640 1 -- 172.21.15.16:0/184977546 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf600327e0 con 0x7fcf74158f30 2024-08-17T02:48:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.708+0000 7fcf7a400640 1 -- 172.21.15.16:0/184977546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf74158f30 msgr2=0x7fcf740a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.708+0000 7fcf7a400640 1 --2- 172.21.15.16:0/184977546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf74158f30 0x7fcf740a98e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009910 tx=0x7fcf600091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.708+0000 7fcf7a400640 1 -- 172.21.15.16:0/184977546 shutdown_connections 2024-08-17T02:48:44.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.708+0000 7fcf7a400640 1 --2- 172.21.15.16:0/184977546 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf74158f30 0x7fcf740a98e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.708+0000 7fcf7a400640 1 -- 172.21.15.16:0/184977546 >> 172.21.15.16:0/184977546 conn(0x7fcf74067750 msgr2=0x7fcf740cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.708+0000 7fcf7a400640 1 -- 172.21.15.16:0/184977546 shutdown_connections 2024-08-17T02:48:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.708+0000 7fcf7a400640 1 -- 172.21.15.16:0/184977546 wait complete. 2024-08-17T02:48:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.710+0000 7fcf7a400640 1 Processor -- start 2024-08-17T02:48:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.710+0000 7fcf7a400640 1 -- start start 2024-08-17T02:48:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf7a400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf740a9e20 0x7fcf7413e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf7a400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf74158b60 0x7fcf7415aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf7a400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf74158f30 0x7fcf74159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf7a400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7413ecc0 con 0x7fcf74158f30 2024-08-17T02:48:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf7a400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7415aa30 con 0x7fcf740a9e20 2024-08-17T02:48:44.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf7a400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74159e20 con 0x7fcf74158b60 2024-08-17T02:48:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf79a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf74158f30 0x7fcf74159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf79a00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf74158f30 0x7fcf74159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39546/0 (socket says 172.21.15.16:39546) 2024-08-17T02:48:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf79a00640 1 -- 172.21.15.16:0/4198200246 learned_addr learned my addr 172.21.15.16:0/4198200246 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf79000640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf740a9e20 0x7fcf7413e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf73e00640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf74158b60 0x7fcf7415aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf79a00640 1 -- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf74158b60 msgr2=0x7fcf7415aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf79a00640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf74158b60 0x7fcf7415aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf79a00640 1 -- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf740a9e20 msgr2=0x7fcf7413e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf79a00640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf740a9e20 0x7fcf7413e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:44.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.711+0000 7fcf79a00640 1 -- 172.21.15.16:0/4198200246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf60009450 con 0x7fcf74158f30 2024-08-17T02:48:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.712+0000 7fcf79a00640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf74158f30 0x7fcf74159410 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00c860 tx=0x7fcf6c0015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.712+0000 7fcf67e00640 1 -- 172.21.15.16:0/4198200246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00ce50 con 0x7fcf74158f30 2024-08-17T02:48:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.712+0000 7fcf67e00640 1 -- 172.21.15.16:0/4198200246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c005d60 con 0x7fcf74158f30 2024-08-17T02:48:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.712+0000 7fcf67e00640 1 -- 172.21.15.16:0/4198200246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00a900 con 0x7fcf74158f30 2024-08-17T02:48:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.712+0000 7fcf7a400640 1 -- 172.21.15.16:0/4198200246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7415a100 con 0x7fcf74158f30 2024-08-17T02:48:44.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.712+0000 7fcf7a400640 1 -- 172.21.15.16:0/4198200246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7415a6c0 con 0x7fcf74158f30 2024-08-17T02:48:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.713+0000 7fcf7a400640 1 -- 172.21.15.16:0/4198200246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7415abc0 con 0x7fcf74158f30 2024-08-17T02:48:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.714+0000 7fcf67e00640 1 -- 172.21.15.16:0/4198200246 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fcf6c00acf0 con 0x7fcf74158f30 2024-08-17T02:48:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.715+0000 7fcf67e00640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcf54065150 0x7fcf54067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.715+0000 7fcf67e00640 1 -- 172.21.15.16:0/4198200246 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5923+0+0 (secure 0 0 0) 0x7fcf6c082370 con 0x7fcf74158f30 2024-08-17T02:48:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.715+0000 7fcf79000640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcf54065150 0x7fcf54067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:44.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.716+0000 7fcf79000640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcf54065150 0x7fcf54067600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcf680037d0 tx=0x7fcf68009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:44.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.719+0000 7fcf67e00640 1 -- 172.21.15.16:0/4198200246 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fcf6c050330 con 0x7fcf74158f30 2024-08-17T02:48:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:44.901+0000 7fcf7a400640 1 -- 172.21.15.16:0/4198200246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000521811.meta", "pool2": "cephfs.volume_0000000000521811.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcf74071050 con 0x7fcf74158f30 2024-08-17T02:48:45.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.644+0000 7fcf67e00640 1 -- 172.21.15.16:0/4198200246 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000521811.meta", "pool2": "cephfs.volume_0000000000521811.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000521811.meta' removed v45) v1 ==== 241+0+0 (secure 0 0 0) 0x7fcf6c053950 con 0x7fcf74158f30 2024-08-17T02:48:45.645 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs.volume_0000000000521811.meta' removed 2024-08-17T02:48:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.648+0000 7fcf7a400640 1 -- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcf54065150 msgr2=0x7fcf54067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.648+0000 7fcf7a400640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcf54065150 0x7fcf54067600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcf680037d0 tx=0x7fcf68009040 comp rx=0 tx=0).stop 2024-08-17T02:48:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.648+0000 7fcf7a400640 1 -- 172.21.15.16:0/4198200246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf74158f30 msgr2=0x7fcf74159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.648+0000 7fcf7a400640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf74158f30 0x7fcf74159410 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00c860 tx=0x7fcf6c0015f0 comp rx=0 tx=0).stop 2024-08-17T02:48:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.648+0000 7fcf7a400640 1 -- 172.21.15.16:0/4198200246 shutdown_connections 2024-08-17T02:48:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.648+0000 7fcf7a400640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fcf54065150 0x7fcf54067600 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:45.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.648+0000 7fcf7a400640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf74158f30 0x7fcf74159410 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.648+0000 7fcf7a400640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fcf74158b60 0x7fcf7415aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.648+0000 7fcf7a400640 1 --2- 172.21.15.16:0/4198200246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf740a9e20 0x7fcf7413e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.648+0000 7fcf7a400640 1 -- 172.21.15.16:0/4198200246 >> 172.21.15.16:0/4198200246 conn(0x7fcf74067750 msgr2=0x7fcf74069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.648+0000 7fcf7a400640 1 -- 172.21.15.16:0/4198200246 shutdown_connections 2024-08-17T02:48:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.649+0000 7fcf7a400640 1 -- 172.21.15.16:0/4198200246 wait complete. 2024-08-17T02:48:45.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000521811.data cephfs.volume_0000000000521811.data --yes-i-really-really-mean-it 2024-08-17T02:48:45.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.832+0000 7f10e0200640 1 Processor -- start 2024-08-17T02:48:45.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.832+0000 7f10e0200640 1 -- start start 2024-08-17T02:48:45.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.833+0000 7f10e0200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f10d80aa810 0x7f10d80aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.833+0000 7f10e0200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d80ac4e0 con 0x7f10d80af7c0 2024-08-17T02:48:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.833+0000 7f10e0200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d80ac780 con 0x7f10d80ab120 2024-08-17T02:48:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.833+0000 7f10e0200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d80aca60 con 0x7f10d80aa810 2024-08-17T02:48:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.834+0000 7f10de200640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f10d80af7c0 0x7f10d8068790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34992/0 (socket says 172.21.15.16:34992) 2024-08-17T02:48:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.834+0000 7f10de200640 1 -- 172.21.15.16:0/1427557073 learned_addr learned my addr 172.21.15.16:0/1427557073 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:45.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.834+0000 7f10dd800640 1 --2- 172.21.15.16:0/1427557073 >> v2:172.21.15.142:3300/0 conn(0x7f10d80aa810 0x7f10d80aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.834+0000 7f10d7e00640 1 -- 172.21.15.16:0/1427557073 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181003708 0 0) 0x7f10d80ac4e0 con 0x7f10d80af7c0 2024-08-17T02:48:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.834+0000 7f10d7e00640 1 -- 172.21.15.16:0/1427557073 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10c00034e0 con 0x7f10d80af7c0 2024-08-17T02:48:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.834+0000 7f10dd800640 1 -- 172.21.15.16:0/1427557073 >> v1:172.21.15.142:6789/0 conn(0x7f10d80ab120 legacy=0x7f10d8068080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.834+0000 7f10dd800640 1 -- 172.21.15.16:0/1427557073 >> v1:172.21.15.16:6789/0 conn(0x7f10d80af7c0 legacy=0x7f10d8068790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.834+0000 7f10dd800640 1 -- 172.21.15.16:0/1427557073 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d80acd40 con 0x7f10d80aa810 2024-08-17T02:48:45.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.836+0000 7f10dd800640 1 --2- 172.21.15.16:0/1427557073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d80aa810 0x7f10d80aabe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009850 tx=0x7f10cc0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81e66fdf77ddc43f server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:45.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.836+0000 7f10d7e00640 1 -- 172.21.15.16:0/1427557073 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc02ae20 con 0x7f10d80aa810 2024-08-17T02:48:45.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.836+0000 7f10d7e00640 1 -- 172.21.15.16:0/1427557073 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10cc032440 con 0x7f10d80aa810 2024-08-17T02:48:45.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.836+0000 7f10d7e00640 1 -- 172.21.15.16:0/1427557073 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc0327c0 con 0x7f10d80aa810 2024-08-17T02:48:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.836+0000 7f10e0200640 1 -- 172.21.15.16:0/1427557073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d80aa810 msgr2=0x7f10d80aabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.836+0000 7f10e0200640 1 --2- 172.21.15.16:0/1427557073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d80aa810 0x7f10d80aabe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009850 tx=0x7f10cc0090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.837+0000 7f10e0200640 1 -- 172.21.15.16:0/1427557073 shutdown_connections 2024-08-17T02:48:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.837+0000 7f10e0200640 1 --2- 172.21.15.16:0/1427557073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d80aa810 0x7f10d80aabe0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.837+0000 7f10e0200640 1 -- 172.21.15.16:0/1427557073 >> 172.21.15.16:0/1427557073 conn(0x7f10d8096a70 msgr2=0x7f10d8096e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.837+0000 7f10e0200640 1 -- 172.21.15.16:0/1427557073 shutdown_connections 2024-08-17T02:48:45.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.837+0000 7f10e0200640 1 -- 172.21.15.16:0/1427557073 wait complete. 2024-08-17T02:48:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.838+0000 7f10e0200640 1 Processor -- start 2024-08-17T02:48:45.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.838+0000 7f10e0200640 1 -- start start 2024-08-17T02:48:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.839+0000 7f10e0200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d80aa810 0x7f10d80982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.839+0000 7f10e0200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d80ab120 0x7f10d809b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.839+0000 7f10e0200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d80af7c0 0x7f10d81403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.839+0000 7f10e0200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d8098810 con 0x7f10d80ab120 2024-08-17T02:48:45.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.839+0000 7f10e0200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d809b810 con 0x7f10d80aa810 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.839+0000 7f10e0200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d809a230 con 0x7f10d80af7c0 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.839+0000 7f10dce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d80ab120 0x7f10d809b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.839+0000 7f10dce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d80ab120 0x7f10d809b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39558/0 (socket says 172.21.15.16:39558) 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.839+0000 7f10dce00640 1 -- 172.21.15.16:0/3306932294 learned_addr learned my addr 172.21.15.16:0/3306932294 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.839+0000 7f10dd800640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d80aa810 0x7f10d80982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.839+0000 7f10de200640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d80af7c0 0x7f10d81403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.840+0000 7f10dce00640 1 -- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d80af7c0 msgr2=0x7f10d81403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.840+0000 7f10dce00640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d80af7c0 0x7f10d81403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.840+0000 7f10dce00640 1 -- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d80aa810 msgr2=0x7f10d80982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.840+0000 7f10dce00640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d80aa810 0x7f10d80982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.840+0000 7f10dce00640 1 -- 172.21.15.16:0/3306932294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10cc009390 con 0x7f10d80ab120 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.840+0000 7f10de200640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d80af7c0 0x7f10d81403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.840+0000 7f10dce00640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d80ab120 0x7f10d809b2d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f10c400c560 tx=0x7f10c4007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:45.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.840+0000 7f10d4c00640 1 -- 172.21.15.16:0/3306932294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4011590 con 0x7f10d80ab120 2024-08-17T02:48:45.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.840+0000 7f10d4c00640 1 -- 172.21.15.16:0/3306932294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c40116f0 con 0x7f10d80ab120 2024-08-17T02:48:45.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.840+0000 7f10d4c00640 1 -- 172.21.15.16:0/3306932294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c4011af0 con 0x7f10d80ab120 2024-08-17T02:48:45.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.840+0000 7f10e0200640 1 -- 172.21.15.16:0/3306932294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d809a510 con 0x7f10d80ab120 2024-08-17T02:48:45.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.840+0000 7f10e0200640 1 -- 172.21.15.16:0/3306932294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d809adb0 con 0x7f10d80ab120 2024-08-17T02:48:45.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.841+0000 7f10e0200640 1 -- 172.21.15.16:0/3306932294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d806ba80 con 0x7f10d80ab120 2024-08-17T02:48:45.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.842+0000 7f10d4c00640 1 -- 172.21.15.16:0/3306932294 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f10c401b3b0 con 0x7f10d80ab120 2024-08-17T02:48:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.843+0000 7f10d4c00640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f10b8065150 0x7f10b8067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.843+0000 7f10d4c00640 1 -- 172.21.15.16:0/3306932294 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5448+0+0 (secure 0 0 0) 0x7f10c4081790 con 0x7f10d80ab120 2024-08-17T02:48:45.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.843+0000 7f10dd800640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f10b8065150 0x7f10b8067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:45.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.844+0000 7f10dd800640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f10b8065150 0x7f10b8067600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10cc034000 tx=0x7f10cc032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:45.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:45.847+0000 7f10d4c00640 1 -- 172.21.15.16:0/3306932294 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f10d806ba80 con 0x7f10d80ab120 2024-08-17T02:48:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.029+0000 7f10e0200640 1 -- 172.21.15.16:0/3306932294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000521811.data", "pool2": "cephfs.volume_0000000000521811.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f10d806ba80 con 0x7f10d80ab120 2024-08-17T02:48:46.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.655+0000 7f10d4c00640 1 -- 172.21.15.16:0/3306932294 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000521811.data", "pool2": "cephfs.volume_0000000000521811.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000521811.data' removed v46) v1 ==== 241+0+0 (secure 0 0 0) 0x7f10d806ba80 con 0x7f10d80ab120 2024-08-17T02:48:46.657 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs.volume_0000000000521811.data' removed 2024-08-17T02:48:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.658+0000 7f10e0200640 1 -- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f10b8065150 msgr2=0x7f10b8067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.658+0000 7f10e0200640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f10b8065150 0x7f10b8067600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10cc034000 tx=0x7f10cc032000 comp rx=0 tx=0).stop 2024-08-17T02:48:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.658+0000 7f10e0200640 1 -- 172.21.15.16:0/3306932294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d80ab120 msgr2=0x7f10d809b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.658+0000 7f10e0200640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d80ab120 0x7f10d809b2d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f10c400c560 tx=0x7f10c4007eb0 comp rx=0 tx=0).stop 2024-08-17T02:48:46.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.658+0000 7f10e0200640 1 -- 172.21.15.16:0/3306932294 shutdown_connections 2024-08-17T02:48:46.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.658+0000 7f10e0200640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f10b8065150 0x7f10b8067600 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:46.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.658+0000 7f10e0200640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f10d80af7c0 0x7f10d81403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:46.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.658+0000 7f10e0200640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10d80ab120 0x7f10d809b2d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:46.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.658+0000 7f10e0200640 1 --2- 172.21.15.16:0/3306932294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f10d80aa810 0x7f10d80982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:46.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.658+0000 7f10e0200640 1 -- 172.21.15.16:0/3306932294 >> 172.21.15.16:0/3306932294 conn(0x7f10d8096a70 msgr2=0x7f10d80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:46.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.658+0000 7f10e0200640 1 -- 172.21.15.16:0/3306932294 shutdown_connections 2024-08-17T02:48:46.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.658+0000 7f10e0200640 1 -- 172.21.15.16:0/3306932294 wait complete. 2024-08-17T02:48:46.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-08-17T02:48:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.834+0000 7f5033600640 1 Processor -- start 2024-08-17T02:48:46.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.834+0000 7f5033600640 1 -- start start 2024-08-17T02:48:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.835+0000 7f5033600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f502c155980 0x7f502c156f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.835+0000 7f5033600640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f502c0af580 0x7f502c0af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.835+0000 7f5033600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502c0aff20 con 0x7f502c0a54d0 2024-08-17T02:48:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.835+0000 7f5033600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502c158400 con 0x7f502c155980 2024-08-17T02:48:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.835+0000 7f5033600640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502c1577f0 con 0x7f502c0af580 2024-08-17T02:48:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.835+0000 7f5030c00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f502c155980 0x7f502c156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.835+0000 7f502be00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f502c0a54d0 0x7f502c155d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35000/0 (socket says 172.21.15.16:35000) 2024-08-17T02:48:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.835+0000 7f502be00640 1 -- 172.21.15.16:0/3254765710 learned_addr learned my addr 172.21.15.16:0/3254765710 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:46.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.836+0000 7f5031600640 1 --2- 172.21.15.16:0/3254765710 >> v2:172.21.15.142:3300/0 conn(0x7f502c0af580 0x7f502c0af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.836+0000 7f5030c00640 1 -- 172.21.15.16:0/3254765710 >> v2:172.21.15.142:3300/0 conn(0x7f502c0af580 msgr2=0x7f502c0af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.836+0000 7f5030c00640 1 --2- 172.21.15.16:0/3254765710 >> v2:172.21.15.142:3300/0 conn(0x7f502c0af580 0x7f502c0af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.836+0000 7f5030c00640 1 -- 172.21.15.16:0/3254765710 >> v1:172.21.15.16:6789/0 conn(0x7f502c0a54d0 legacy=0x7f502c155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.836+0000 7f5030c00640 1 -- 172.21.15.16:0/3254765710 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c157ad0 con 0x7f502c155980 2024-08-17T02:48:46.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.836+0000 7f502b400640 1 -- 172.21.15.16:0/3254765710 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253024055 0 0) 0x7f5018001110 con 0x7f502c0a54d0 2024-08-17T02:48:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.837+0000 7f5030c00640 1 --2- 172.21.15.16:0/3254765710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c155980 0x7f502c156f70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f502c06b700 tx=0x7f50200090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b41d1348f4c9bb server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.837+0000 7f502b400640 1 -- 172.21.15.16:0/3254765710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502002adb0 con 0x7f502c155980 2024-08-17T02:48:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.837+0000 7f502b400640 1 -- 172.21.15.16:0/3254765710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5020032440 con 0x7f502c155980 2024-08-17T02:48:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.837+0000 7f502b400640 1 -- 172.21.15.16:0/3254765710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50200327e0 con 0x7f502c155980 2024-08-17T02:48:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.838+0000 7f5033600640 1 -- 172.21.15.16:0/3254765710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c155980 msgr2=0x7f502c156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.838+0000 7f5033600640 1 --2- 172.21.15.16:0/3254765710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c155980 0x7f502c156f70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f502c06b700 tx=0x7f50200090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:46.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.838+0000 7f5033600640 1 -- 172.21.15.16:0/3254765710 shutdown_connections 2024-08-17T02:48:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.838+0000 7f5033600640 1 --2- 172.21.15.16:0/3254765710 >> v2:172.21.15.142:3300/0 conn(0x7f502c0af580 0x7f502c0af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.838+0000 7f5033600640 1 --2- 172.21.15.16:0/3254765710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c155980 0x7f502c156f70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.838+0000 7f5033600640 1 -- 172.21.15.16:0/3254765710 >> 172.21.15.16:0/3254765710 conn(0x7f502c067750 msgr2=0x7f502c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.838+0000 7f5033600640 1 -- 172.21.15.16:0/3254765710 shutdown_connections 2024-08-17T02:48:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.838+0000 7f5033600640 1 -- 172.21.15.16:0/3254765710 wait complete. 2024-08-17T02:48:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.839+0000 7f5033600640 1 Processor -- start 2024-08-17T02:48:46.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.839+0000 7f5033600640 1 -- start start 2024-08-17T02:48:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.840+0000 7f5033600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f502c0a54d0 0x7f502c16a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.840+0000 7f5033600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0af580 0x7f502c14eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.840+0000 7f5033600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f502c155980 0x7f502c169150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.840+0000 7f5033600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c0c34a0 con 0x7f502c0af580 2024-08-17T02:48:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.840+0000 7f5033600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c169a40 con 0x7f502c155980 2024-08-17T02:48:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.840+0000 7f5033600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c169d20 con 0x7f502c0a54d0 2024-08-17T02:48:46.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.841+0000 7f502be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0af580 0x7f502c14eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.841+0000 7f502be00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0af580 0x7f502c14eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39570/0 (socket says 172.21.15.16:39570) 2024-08-17T02:48:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.841+0000 7f502be00640 1 -- 172.21.15.16:0/381239916 learned_addr learned my addr 172.21.15.16:0/381239916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.841+0000 7f5030c00640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f502c0a54d0 0x7f502c16a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.841+0000 7f5031600640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f502c155980 0x7f502c169150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.841+0000 7f502be00640 1 -- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f502c0a54d0 msgr2=0x7f502c16a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.841+0000 7f502be00640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f502c0a54d0 0x7f502c16a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.841+0000 7f502be00640 1 -- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f502c155980 msgr2=0x7f502c169150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.841+0000 7f502be00640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f502c155980 0x7f502c169150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.841+0000 7f502be00640 1 -- 172.21.15.16:0/381239916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5020009390 con 0x7f502c0af580 2024-08-17T02:48:46.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.841+0000 7f502be00640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0af580 0x7f502c14eff0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f501800c510 tx=0x7f501800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.842+0000 7f501fe00640 1 -- 172.21.15.16:0/381239916 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018005aa0 con 0x7f502c0af580 2024-08-17T02:48:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.842+0000 7f501fe00640 1 -- 172.21.15.16:0/381239916 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5018005c20 con 0x7f502c0af580 2024-08-17T02:48:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.842+0000 7f501fe00640 1 -- 172.21.15.16:0/381239916 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50180070d0 con 0x7f502c0af580 2024-08-17T02:48:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.842+0000 7f5033600640 1 -- 172.21.15.16:0/381239916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c16a000 con 0x7f502c0af580 2024-08-17T02:48:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.842+0000 7f5033600640 1 -- 172.21.15.16:0/381239916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c16a850 con 0x7f502c0af580 2024-08-17T02:48:46.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.843+0000 7f5033600640 1 -- 172.21.15.16:0/381239916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502c0c0dc0 con 0x7f502c0af580 2024-08-17T02:48:46.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.844+0000 7f501fe00640 1 -- 172.21.15.16:0/381239916 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f50180074c0 con 0x7f502c0af580 2024-08-17T02:48:46.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.844+0000 7f501fe00640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f500c065150 0x7f500c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:46.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.844+0000 7f501fe00640 1 -- 172.21.15.16:0/381239916 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f5018081430 con 0x7f502c0af580 2024-08-17T02:48:46.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.845+0000 7f5030c00640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f500c065150 0x7f500c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:46.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.846+0000 7f5030c00640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f500c065150 0x7f500c067600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5020034000 tx=0x7f5020032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:46.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:46.848+0000 7f501fe00640 1 -- 172.21.15.16:0/381239916 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f501804f7a0 con 0x7f502c0af580 2024-08-17T02:48:47.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.030+0000 7f5033600640 1 -- 172.21.15.16:0/381239916 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f502c0aff20 con 0x7f502c0af580 2024-08-17T02:48:47.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.034+0000 7f501fe00640 1 -- 172.21.15.16:0/381239916 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f502c0aff20 con 0x7f502c0af580 2024-08-17T02:48:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.036+0000 7f5033600640 1 -- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f500c065150 msgr2=0x7f500c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.036+0000 7f5033600640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f500c065150 0x7f500c067600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5020034000 tx=0x7f5020032000 comp rx=0 tx=0).stop 2024-08-17T02:48:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.036+0000 7f5033600640 1 -- 172.21.15.16:0/381239916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0af580 msgr2=0x7f502c14eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.036+0000 7f5033600640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0af580 0x7f502c14eff0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f501800c510 tx=0x7f501800c910 comp rx=0 tx=0).stop 2024-08-17T02:48:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.036+0000 7f5033600640 1 -- 172.21.15.16:0/381239916 shutdown_connections 2024-08-17T02:48:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.036+0000 7f5033600640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f500c065150 0x7f500c067600 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.036+0000 7f5033600640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f502c155980 0x7f502c169150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.036+0000 7f5033600640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f502c0af580 0x7f502c14eff0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.036+0000 7f5033600640 1 --2- 172.21.15.16:0/381239916 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f502c0a54d0 0x7f502c16a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.036+0000 7f5033600640 1 -- 172.21.15.16:0/381239916 >> 172.21.15.16:0/381239916 conn(0x7f502c067750 msgr2=0x7f502c069890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:47.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.036+0000 7f5033600640 1 -- 172.21.15.16:0/381239916 shutdown_connections 2024-08-17T02:48:47.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.036+0000 7f5033600640 1 -- 172.21.15.16:0/381239916 wait complete. 2024-08-17T02:48:47.053 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2024-08-17T02:48:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.203+0000 7fa79f000640 1 Processor -- start 2024-08-17T02:48:47.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.203+0000 7fa79f000640 1 -- start start 2024-08-17T02:48:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.204+0000 7fa79f000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa798159ca0 0x7fa7980ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.204+0000 7fa79f000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa7980ca8a0 0x7fa7980cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.204+0000 7fa79f000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7980cadc0 con 0x7fa7980ccad0 2024-08-17T02:48:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.204+0000 7fa79f000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7980cd6e0 con 0x7fa7980ca8a0 2024-08-17T02:48:47.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.204+0000 7fa79f000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980ccf70 con 0x7fa798159ca0 2024-08-17T02:48:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.205+0000 7fa79d200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa7980ca8a0 0x7fa7980cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.205+0000 7fa79dc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa798159ca0 0x7fa7980ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.205+0000 7fa79d200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa7980ca8a0 0x7fa7980cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.16:46158/0 (socket says 172.21.15.16:46158) 2024-08-17T02:48:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.205+0000 7fa79e600640 1 --1- >> v1:172.21.15.142:6789/0 conn(0x7fa7980ccad0 0x7fa7980cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:35278/0 (socket says 172.21.15.16:35278) 2024-08-17T02:48:47.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.205+0000 7fa79d200640 1 -- 172.21.15.16:0/3276317449 learned_addr learned my addr 172.21.15.16:0/3276317449 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:47.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.205+0000 7fa79dc00640 1 -- 172.21.15.16:0/3276317449 >> v2:172.21.15.142:3300/0 conn(0x7fa7980ca8a0 msgr2=0x7fa7980cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:47.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.205+0000 7fa79dc00640 1 --2- 172.21.15.16:0/3276317449 >> v2:172.21.15.142:3300/0 conn(0x7fa7980ca8a0 0x7fa7980cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.205+0000 7fa79dc00640 1 -- 172.21.15.16:0/3276317449 >> v1:172.21.15.142:6789/0 conn(0x7fa7980ccad0 legacy=0x7fa7980cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:47.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.205+0000 7fa79dc00640 1 -- 172.21.15.16:0/3276317449 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7980cb060 con 0x7fa798159ca0 2024-08-17T02:48:47.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.205+0000 7fa79d200640 1 --2- 172.21.15.16:0/3276317449 >> v2:172.21.15.142:3300/0 conn(0x7fa7980ca8a0 0x7fa7980cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:48:47.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.206+0000 7fa79dc00640 1 --2- 172.21.15.16:0/3276317449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa798159ca0 0x7fa7980ca360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009850 tx=0x7fa78c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbe9d79b1af00575 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:47.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.207+0000 7fa797e00640 1 -- 172.21.15.16:0/3276317449 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c02ad90 con 0x7fa798159ca0 2024-08-17T02:48:47.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.207+0000 7fa797e00640 1 -- 172.21.15.16:0/3276317449 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c032440 con 0x7fa798159ca0 2024-08-17T02:48:47.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.207+0000 7fa797e00640 1 -- 172.21.15.16:0/3276317449 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c032820 con 0x7fa798159ca0 2024-08-17T02:48:47.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.207+0000 7fa79f000640 1 -- 172.21.15.16:0/3276317449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa798159ca0 msgr2=0x7fa7980ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:47.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.207+0000 7fa79f000640 1 --2- 172.21.15.16:0/3276317449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa798159ca0 0x7fa7980ca360 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa78c009850 tx=0x7fa78c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.207+0000 7fa79f000640 1 -- 172.21.15.16:0/3276317449 shutdown_connections 2024-08-17T02:48:47.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.207+0000 7fa79f000640 1 --2- 172.21.15.16:0/3276317449 >> v2:172.21.15.142:3300/0 conn(0x7fa7980ca8a0 0x7fa7980cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.207+0000 7fa79f000640 1 --2- 172.21.15.16:0/3276317449 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa798159ca0 0x7fa7980ca360 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.207+0000 7fa79f000640 1 -- 172.21.15.16:0/3276317449 >> 172.21.15.16:0/3276317449 conn(0x7fa798098e00 msgr2=0x7fa79806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:47.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.207+0000 7fa79f000640 1 -- 172.21.15.16:0/3276317449 shutdown_connections 2024-08-17T02:48:47.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.207+0000 7fa79f000640 1 -- 172.21.15.16:0/3276317449 wait complete. 2024-08-17T02:48:47.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.209+0000 7fa79f000640 1 Processor -- start 2024-08-17T02:48:47.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.209+0000 7fa79f000640 1 -- start start 2024-08-17T02:48:47.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.210+0000 7fa79f000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7980ca8a0 0x7fa798099930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:47.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.210+0000 7fa79f000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7980ccad0 0x7fa79809b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:47.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.210+0000 7fa79f000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa798159ca0 0x7fa79809b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:47.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.210+0000 7fa79f000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980c9720 con 0x7fa7980ccad0 2024-08-17T02:48:47.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.210+0000 7fa79f000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa798168b80 con 0x7fa7980ca8a0 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.210+0000 7fa79f000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980c8b10 con 0x7fa798159ca0 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.210+0000 7fa79d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7980ccad0 0x7fa79809b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.210+0000 7fa79d200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7980ccad0 0x7fa79809b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39584/0 (socket says 172.21.15.16:39584) 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.210+0000 7fa79d200640 1 -- 172.21.15.16:0/2954115207 learned_addr learned my addr 172.21.15.16:0/2954115207 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.210+0000 7fa79dc00640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7980ca8a0 0x7fa798099930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.210+0000 7fa79e600640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa798159ca0 0x7fa79809b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.211+0000 7fa79d200640 1 -- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa798159ca0 msgr2=0x7fa79809b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.211+0000 7fa79d200640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa798159ca0 0x7fa79809b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.211+0000 7fa79d200640 1 -- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7980ca8a0 msgr2=0x7fa798099930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.211+0000 7fa79d200640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7980ca8a0 0x7fa798099930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.211+0000 7fa79d200640 1 -- 172.21.15.16:0/2954115207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa784009450 con 0x7fa7980ccad0 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.211+0000 7fa79e600640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa798159ca0 0x7fa79809b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:47.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.211+0000 7fa79d200640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7980ccad0 0x7fa79809b410 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa78400ec80 tx=0x7fa7840091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.211+0000 7fa794c00640 1 -- 172.21.15.16:0/2954115207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78400eec0 con 0x7fa7980ccad0 2024-08-17T02:48:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.211+0000 7fa794c00640 1 -- 172.21.15.16:0/2954115207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78400cbc0 con 0x7fa7980ccad0 2024-08-17T02:48:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.211+0000 7fa794c00640 1 -- 172.21.15.16:0/2954115207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784013470 con 0x7fa7980ccad0 2024-08-17T02:48:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.211+0000 7fa79f000640 1 -- 172.21.15.16:0/2954115207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78c009390 con 0x7fa7980ccad0 2024-08-17T02:48:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.211+0000 7fa79f000640 1 -- 172.21.15.16:0/2954115207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7980c93b0 con 0x7fa7980ccad0 2024-08-17T02:48:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.212+0000 7fa79f000640 1 -- 172.21.15.16:0/2954115207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7980c3d00 con 0x7fa7980ccad0 2024-08-17T02:48:47.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.213+0000 7fa794c00640 1 -- 172.21.15.16:0/2954115207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fa784013820 con 0x7fa7980ccad0 2024-08-17T02:48:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.214+0000 7fa794c00640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa778065150 0x7fa778067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.215+0000 7fa794c00640 1 -- 172.21.15.16:0/2954115207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fa7840825a0 con 0x7fa7980ccad0 2024-08-17T02:48:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.215+0000 7fa79dc00640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa778065150 0x7fa778067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:47.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.216+0000 7fa79dc00640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa778065150 0x7fa778067600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa78c034000 tx=0x7fa78c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:47.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.218+0000 7fa794c00640 1 -- 172.21.15.16:0/2954115207 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa784050910 con 0x7fa7980ccad0 2024-08-17T02:48:47.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.469+0000 7fa79f000640 1 -- 172.21.15.16:0/2954115207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7fa798071050 con 0x7fa7980ccad0 2024-08-17T02:48:47.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.657+0000 7fa794c00640 1 -- 172.21.15.16:0/2954115207 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v163) v1 ==== 135+0+0 (secure 0 0 0) 0x7fa784053f30 con 0x7fa7980ccad0 2024-08-17T02:48:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.659+0000 7fa79f000640 1 -- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa778065150 msgr2=0x7fa778067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:47.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.659+0000 7fa79f000640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa778065150 0x7fa778067600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa78c034000 tx=0x7fa78c032000 comp rx=0 tx=0).stop 2024-08-17T02:48:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.659+0000 7fa79f000640 1 -- 172.21.15.16:0/2954115207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7980ccad0 msgr2=0x7fa79809b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.659+0000 7fa79f000640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7980ccad0 0x7fa79809b410 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa78400ec80 tx=0x7fa7840091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.659+0000 7fa79f000640 1 -- 172.21.15.16:0/2954115207 shutdown_connections 2024-08-17T02:48:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.659+0000 7fa79f000640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa778065150 0x7fa778067600 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.659+0000 7fa79f000640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa798159ca0 0x7fa79809b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.659+0000 7fa79f000640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7980ccad0 0x7fa79809b410 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.659+0000 7fa79f000640 1 --2- 172.21.15.16:0/2954115207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7980ca8a0 0x7fa798099930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.659+0000 7fa79f000640 1 -- 172.21.15.16:0/2954115207 >> 172.21.15.16:0/2954115207 conn(0x7fa798098e00 msgr2=0x7fa798157020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.659+0000 7fa79f000640 1 -- 172.21.15.16:0/2954115207 shutdown_connections 2024-08-17T02:48:47.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:47.659+0000 7fa79f000640 1 -- 172.21.15.16:0/2954115207 wait complete. 2024-08-17T02:48:47.679 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ERROR 2024-08-17T02:48:47.680 INFO:tasks.cephfs_test_runner: 2024-08-17T02:48:47.680 INFO:tasks.cephfs_test_runner:====================================================================== 2024-08-17T02:48:47.680 INFO:tasks.cephfs_test_runner:ERROR: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-17T02:48:47.680 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up 2024-08-17T02:48:47.680 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-17T02:48:47.680 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-08-17T02:48:47.680 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/cephfs/test_volumes.py", line 459, in test_volume_create 2024-08-17T02:48:47.680 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", volname, "--yes-i-really-mean-it") 2024-08-17T02:48:47.680 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2024-08-17T02:48:47.680 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/ceph_manager.py", line 1631, in run_cluster_cmd 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: r.wait() 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000521811 --yes-i-really-mean-it' 2024-08-17T02:48:47.681 INFO:tasks.cephfs_test_runner: 2024-08-17T02:48:47.682 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-17T02:48:47.682 INFO:tasks.cephfs_test_runner:Ran 1 test in 176.504s 2024-08-17T02:48:47.682 INFO:tasks.cephfs_test_runner: 2024-08-17T02:48:47.682 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-08-17T02:48:47.682 INFO:tasks.cephfs_test_runner: 2024-08-17T02:48:47.682 INFO:tasks.cephfs_test_runner:====================================================================== 2024-08-17T02:48:47.682 INFO:tasks.cephfs_test_runner:ERROR: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-17T02:48:47.682 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up 2024-08-17T02:48:47.682 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-08-17T02:48:47.682 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-08-17T02:48:47.682 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/cephfs/test_volumes.py", line 459, in test_volume_create 2024-08-17T02:48:47.682 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", volname, "--yes-i-really-mean-it") 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: return self.get_ceph_cmd_stdout("fs", *args) 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/ceph_manager.py", line 1631, in run_cluster_cmd 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: r.wait() 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-08-17T02:48:47.683 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-08-17T02:48:47.684 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000521811 --yes-i-really-mean-it' 2024-08-17T02:48:47.684 INFO:tasks.cephfs_test_runner: 2024-08-17T02:48:47.684 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-17T02:48:47.914 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1b4aff9bde9c42f09eba9cae6de3ac52 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_2577e8139c6b8786f9d6d6ea21b8da9244b9260e/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2024-08-17T02:48:47.915 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-08-17T02:48:47.924 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-08-17T02:48:47.933 INFO:tasks.kclient:Unmounting kernel clients... 2024-08-17T02:48:47.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:47.934 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-08-17T02:48:47.973 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-17T02:48:47.973 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-08-17T02:48:47.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:47.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-17T02:48:48.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:48.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-08-17T02:48:48.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:48.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-08-17T02:48:48.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:48.620 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-08-17T02:48:48.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:48.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-08-17T02:48:48.757 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-08-17T02:48:48.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:48:48.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:48.760 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-08-17T02:48:48.800 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-08-17T02:48:48.801 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-08-17T02:48:48.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:48.801 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-08-17T02:48:48.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:48.903 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-08-17T02:48:49.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:49.138 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2024-08-17T02:48:49.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:49.438 DEBUG:teuthology.orchestra.run.smithi142:> ip netns list 2024-08-17T02:48:49.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-17T02:48:49.474 DEBUG:teuthology.orchestra.run.smithi142:> sudo ip link delete ceph-brx 2024-08-17T02:48:49.574 INFO:teuthology.orchestra.run.smithi142.stderr:Cannot find device "ceph-brx" 2024-08-17T02:48:49.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:48:49.577 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-08-17T02:48:49.588 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-08-17T02:48:49.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:48:49.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.749+0000 7eff12800640 1 Processor -- start 2024-08-17T02:48:49.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.749+0000 7eff12800640 1 -- start start 2024-08-17T02:48:49.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.750+0000 7eff12800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7eff0c155a20 0x7eff0c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:49.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.750+0000 7eff12800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff0c06df00 0x7eff0c0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:49.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.750+0000 7eff12800640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c156670 con 0x7eff0c155df0 2024-08-17T02:48:49.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.750+0000 7eff12800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0c0b0350 con 0x7eff0c155a20 2024-08-17T02:48:49.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.750+0000 7eff12800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c06e5e0 con 0x7eff0c06df00 2024-08-17T02:48:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.750+0000 7eff0b400640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7eff0c155a20 0x7eff0c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.750+0000 7eff0be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff0c06df00 0x7eff0c0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:49.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.751+0000 7eff0be00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff0c06df00 0x7eff0c0ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:45652/0 (socket says 172.21.15.16:45652) 2024-08-17T02:48:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.751+0000 7eff0be00640 1 -- 172.21.15.16:0/2187235264 learned_addr learned my addr 172.21.15.16:0/2187235264 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.751+0000 7eff03e00640 1 --1- 172.21.15.16:0/2187235264 >> v1:172.21.15.142:6789/0 conn(0x7eff0c155df0 0x7eff0c06d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.142:6789/0 says I am v1:172.21.15.16:35292/0 (socket says 172.21.15.16:35292) 2024-08-17T02:48:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.751+0000 7eff0be00640 1 -- 172.21.15.16:0/2187235264 >> v2:172.21.15.142:3300/0 conn(0x7eff0c155a20 msgr2=0x7eff0c0adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.751+0000 7eff0be00640 1 --2- 172.21.15.16:0/2187235264 >> v2:172.21.15.142:3300/0 conn(0x7eff0c155a20 0x7eff0c0adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.751+0000 7eff0be00640 1 -- 172.21.15.16:0/2187235264 >> v1:172.21.15.142:6789/0 conn(0x7eff0c155df0 legacy=0x7eff0c06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.751+0000 7eff0be00640 1 -- 172.21.15.16:0/2187235264 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c06e8c0 con 0x7eff0c06df00 2024-08-17T02:48:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.752+0000 7eff0be00640 1 --2- 172.21.15.16:0/2187235264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff0c06df00 0x7eff0c0ad9b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7eff04009f20 tx=0x7eff04009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bc8d712ee1e9b4c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.752+0000 7eff0aa00640 1 -- 172.21.15.16:0/2187235264 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04033410 con 0x7eff0c06df00 2024-08-17T02:48:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.752+0000 7eff0aa00640 1 -- 172.21.15.16:0/2187235264 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff04033570 con 0x7eff0c06df00 2024-08-17T02:48:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.752+0000 7eff0aa00640 1 -- 172.21.15.16:0/2187235264 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04033950 con 0x7eff0c06df00 2024-08-17T02:48:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.753+0000 7eff12800640 1 -- 172.21.15.16:0/2187235264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff0c06df00 msgr2=0x7eff0c0ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.753+0000 7eff12800640 1 --2- 172.21.15.16:0/2187235264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff0c06df00 0x7eff0c0ad9b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7eff04009f20 tx=0x7eff04009840 comp rx=0 tx=0).stop 2024-08-17T02:48:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.753+0000 7eff12800640 1 -- 172.21.15.16:0/2187235264 shutdown_connections 2024-08-17T02:48:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.753+0000 7eff12800640 1 --2- 172.21.15.16:0/2187235264 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff0c06df00 0x7eff0c0ad9b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.753+0000 7eff12800640 1 --2- 172.21.15.16:0/2187235264 >> v2:172.21.15.142:3300/0 conn(0x7eff0c155a20 0x7eff0c0adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.753+0000 7eff12800640 1 -- 172.21.15.16:0/2187235264 >> 172.21.15.16:0/2187235264 conn(0x7eff0c0cb9b0 msgr2=0x7eff0c0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.753+0000 7eff12800640 1 -- 172.21.15.16:0/2187235264 shutdown_connections 2024-08-17T02:48:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.753+0000 7eff12800640 1 -- 172.21.15.16:0/2187235264 wait complete. 2024-08-17T02:48:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.754+0000 7eff12800640 1 Processor -- start 2024-08-17T02:48:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.755+0000 7eff12800640 1 -- start start 2024-08-17T02:48:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.755+0000 7eff12800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff0c06df00 0x7eff0c148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.755+0000 7eff12800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c155a20 0x7eff0c14cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.755+0000 7eff12800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff0c155df0 0x7eff0c150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.755+0000 7eff12800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c148b40 con 0x7eff0c155a20 2024-08-17T02:48:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.755+0000 7eff12800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c14f2d0 con 0x7eff0c155df0 2024-08-17T02:48:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.755+0000 7eff12800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c14e400 con 0x7eff0c06df00 2024-08-17T02:48:49.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.756+0000 7eff03e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c155a20 0x7eff0c14cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.756+0000 7eff03e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c155a20 0x7eff0c14cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39600/0 (socket says 172.21.15.16:39600) 2024-08-17T02:48:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.756+0000 7eff03e00640 1 -- 172.21.15.16:0/1531188695 learned_addr learned my addr 172.21.15.16:0/1531188695 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.756+0000 7eff0b400640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff0c06df00 0x7eff0c148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.756+0000 7eff0be00640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff0c155df0 0x7eff0c150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.756+0000 7eff03e00640 1 -- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff0c06df00 msgr2=0x7eff0c148600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.756+0000 7eff03e00640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff0c06df00 0x7eff0c148600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.756+0000 7eff03e00640 1 -- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff0c155df0 msgr2=0x7eff0c150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.756+0000 7eff03e00640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff0c155df0 0x7eff0c150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.756+0000 7eff03e00640 1 -- 172.21.15.16:0/1531188695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff04009ae0 con 0x7eff0c155a20 2024-08-17T02:48:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.757+0000 7eff03e00640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c155a20 0x7eff0c14cee0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efef400c590 tx=0x7efef4007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.757+0000 7eff02a00640 1 -- 172.21.15.16:0/1531188695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4007300 con 0x7eff0c155a20 2024-08-17T02:48:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.757+0000 7eff02a00640 1 -- 172.21.15.16:0/1531188695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef40074a0 con 0x7eff0c155a20 2024-08-17T02:48:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.757+0000 7eff02a00640 1 -- 172.21.15.16:0/1531188695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef40047a0 con 0x7eff0c155a20 2024-08-17T02:48:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.757+0000 7eff12800640 1 -- 172.21.15.16:0/1531188695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0c14e6c0 con 0x7eff0c155a20 2024-08-17T02:48:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.757+0000 7eff12800640 1 -- 172.21.15.16:0/1531188695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0c14ec80 con 0x7eff0c155a20 2024-08-17T02:48:49.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.758+0000 7eff12800640 1 -- 172.21.15.16:0/1531188695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0c064db0 con 0x7eff0c155a20 2024-08-17T02:48:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.759+0000 7eff02a00640 1 -- 172.21.15.16:0/1531188695 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7efef4004bf0 con 0x7eff0c155a20 2024-08-17T02:48:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.760+0000 7eff02a00640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efee8065180 0x7efee8067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:49.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.760+0000 7eff02a00640 1 -- 172.21.15.16:0/1531188695 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7efef4081670 con 0x7eff0c155a20 2024-08-17T02:48:49.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.760+0000 7eff0b400640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efee8065180 0x7efee8067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:49.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.761+0000 7eff0b400640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efee8065180 0x7efee8067630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0cb240 tx=0x7efefc0090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:49.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.763+0000 7eff02a00640 1 -- 172.21.15.16:0/1531188695 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7efef404fb20 con 0x7eff0c155a20 2024-08-17T02:48:49.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.947+0000 7eff12800640 1 -- 172.21.15.16:0/1531188695 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff0c0adab0 con 0x7eff0c155a20 2024-08-17T02:48:49.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.948+0000 7eff02a00640 1 -- 172.21.15.16:0/1531188695 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+12391 (secure 0 0 0) 0x7efef4053140 con 0x7eff0c155a20 2024-08-17T02:48:49.949 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:49.950 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:48:46.649763+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/3115083427":"2024-08-18T02:48:43.481423+0000","172.21.15.16:6832/3115083427":"2024-08-18T02:48:43.481423+0000","172.21.15.16:6835/219684854":"2024-08-18T02:48:36.607854+0000","172.21.15.16:6833/219684854":"2024-08-18T02:48:36.607854+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:48:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.950+0000 7eff12800640 1 -- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efee8065180 msgr2=0x7efee8067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.950+0000 7eff12800640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efee8065180 0x7efee8067630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0cb240 tx=0x7efefc0090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.950+0000 7eff12800640 1 -- 172.21.15.16:0/1531188695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c155a20 msgr2=0x7eff0c14cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:49.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.950+0000 7eff12800640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c155a20 0x7eff0c14cee0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efef400c590 tx=0x7efef4007eb0 comp rx=0 tx=0).stop 2024-08-17T02:48:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.951+0000 7eff12800640 1 -- 172.21.15.16:0/1531188695 shutdown_connections 2024-08-17T02:48:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.951+0000 7eff12800640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7efee8065180 0x7efee8067630 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.951+0000 7eff12800640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff0c155df0 0x7eff0c150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.951+0000 7eff12800640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c155a20 0x7eff0c14cee0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.951+0000 7eff12800640 1 --2- 172.21.15.16:0/1531188695 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7eff0c06df00 0x7eff0c148600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.951+0000 7eff12800640 1 -- 172.21.15.16:0/1531188695 >> 172.21.15.16:0/1531188695 conn(0x7eff0c0cb9b0 msgr2=0x7eff0c157fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.951+0000 7eff12800640 1 -- 172.21.15.16:0/1531188695 shutdown_connections 2024-08-17T02:48:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:49.951+0000 7eff12800640 1 -- 172.21.15.16:0/1531188695 wait complete. 2024-08-17T02:48:49.968 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-08-17T02:48:49.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:48:50.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.117+0000 7f21ee000640 1 Processor -- start 2024-08-17T02:48:50.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.117+0000 7f21ee000640 1 -- start start 2024-08-17T02:48:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.118+0000 7f21ee000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21e8155180 0x7f21e80aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21ee000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e8155550 0x7f21e80ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21ee000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f21e806e080 0x7f21e806e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21ee000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e80afc10 con 0x7f21e8155180 2024-08-17T02:48:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21ee000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e806ea20 con 0x7f21e806e080 2024-08-17T02:48:50.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21ee000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80ae5b0 con 0x7f21e8155550 2024-08-17T02:48:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21e7400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21e8155180 0x7f21e80aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21e7400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21e8155180 0x7f21e80aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39612/0 (socket says 172.21.15.16:39612) 2024-08-17T02:48:50.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21e7400640 1 -- 172.21.15.16:0/1845541371 learned_addr learned my addr 172.21.15.16:0/1845541371 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21e7e00640 1 --2- 172.21.15.16:0/1845541371 >> v2:172.21.15.142:3300/0 conn(0x7f21e806e080 0x7f21e806e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21e6a00640 1 --2- 172.21.15.16:0/1845541371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e8155550 0x7f21e80ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21e7400640 1 -- 172.21.15.16:0/1845541371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e8155550 msgr2=0x7f21e80ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21e7400640 1 --2- 172.21.15.16:0/1845541371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e8155550 0x7f21e80ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21e7400640 1 -- 172.21.15.16:0/1845541371 >> v2:172.21.15.142:3300/0 conn(0x7f21e806e080 msgr2=0x7f21e806e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21e7400640 1 --2- 172.21.15.16:0/1845541371 >> v2:172.21.15.142:3300/0 conn(0x7f21e806e080 0x7f21e806e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21e7400640 1 -- 172.21.15.16:0/1845541371 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80af000 con 0x7f21e8155180 2024-08-17T02:48:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21e7e00640 1 --2- 172.21.15.16:0/1845541371 >> v2:172.21.15.142:3300/0 conn(0x7f21e806e080 0x7f21e806e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.119+0000 7f21e6a00640 1 --2- 172.21.15.16:0/1845541371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e8155550 0x7f21e80ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.120+0000 7f21e7400640 1 --2- 172.21.15.16:0/1845541371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8155180 0x7f21e80aea20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f21e806c770 tx=0x7f21d80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc674e34f2e51c7a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.121+0000 7f21e6000640 1 -- 172.21.15.16:0/1845541371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d802add0 con 0x7f21e8155180 2024-08-17T02:48:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.121+0000 7f21e6000640 1 -- 172.21.15.16:0/1845541371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d8032440 con 0x7f21e8155180 2024-08-17T02:48:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.121+0000 7f21e6000640 1 -- 172.21.15.16:0/1845541371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8032820 con 0x7f21e8155180 2024-08-17T02:48:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.121+0000 7f21ee000640 1 -- 172.21.15.16:0/1845541371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8155180 msgr2=0x7f21e80aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.121+0000 7f21ee000640 1 --2- 172.21.15.16:0/1845541371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8155180 0x7f21e80aea20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f21e806c770 tx=0x7f21d80091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.121+0000 7f21ee000640 1 -- 172.21.15.16:0/1845541371 shutdown_connections 2024-08-17T02:48:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.121+0000 7f21ee000640 1 --2- 172.21.15.16:0/1845541371 >> v2:172.21.15.142:3300/0 conn(0x7f21e806e080 0x7f21e806e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.121+0000 7f21ee000640 1 --2- 172.21.15.16:0/1845541371 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e8155550 0x7f21e80ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.121+0000 7f21ee000640 1 --2- 172.21.15.16:0/1845541371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8155180 0x7f21e80aea20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.121+0000 7f21ee000640 1 -- 172.21.15.16:0/1845541371 >> 172.21.15.16:0/1845541371 conn(0x7f21e80cb9b0 msgr2=0x7f21e80cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.121+0000 7f21ee000640 1 -- 172.21.15.16:0/1845541371 shutdown_connections 2024-08-17T02:48:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.121+0000 7f21ee000640 1 -- 172.21.15.16:0/1845541371 wait complete. 2024-08-17T02:48:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.123+0000 7f21ee000640 1 Processor -- start 2024-08-17T02:48:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.123+0000 7f21ee000640 1 -- start start 2024-08-17T02:48:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.124+0000 7f21ee000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e806e080 0x7f21e814c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.124+0000 7f21ee000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21e8155180 0x7f21e8152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.124+0000 7f21ee000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8155550 0x7f21e81538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.124+0000 7f21ee000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e8152bb0 con 0x7f21e8155550 2024-08-17T02:48:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.124+0000 7f21ee000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e8153e00 con 0x7f21e8155180 2024-08-17T02:48:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.124+0000 7f21ee000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e81691d0 con 0x7f21e806e080 2024-08-17T02:48:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.124+0000 7f21e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8155550 0x7f21e81538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.124+0000 7f21e7e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8155550 0x7f21e81538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39616/0 (socket says 172.21.15.16:39616) 2024-08-17T02:48:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.124+0000 7f21e7e00640 1 -- 172.21.15.16:0/4277577814 learned_addr learned my addr 172.21.15.16:0/4277577814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.124+0000 7f21e6a00640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21e8155180 0x7f21e8152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.124+0000 7f21e7400640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e806e080 0x7f21e814c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.125+0000 7f21e7e00640 1 -- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e806e080 msgr2=0x7f21e814c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.125+0000 7f21e7e00640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e806e080 0x7f21e814c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.125+0000 7f21e7e00640 1 -- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21e8155180 msgr2=0x7f21e8152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.125+0000 7f21e7e00640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21e8155180 0x7f21e8152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.125+0000 7f21e7e00640 1 -- 172.21.15.16:0/4277577814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8009450 con 0x7f21e8155550 2024-08-17T02:48:50.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.125+0000 7f21e7e00640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8155550 0x7f21e81538c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f21e000cf60 tx=0x7f21e0007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.125+0000 7f21dea00640 1 -- 172.21.15.16:0/4277577814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e000ae90 con 0x7f21e8155550 2024-08-17T02:48:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.125+0000 7f21dea00640 1 -- 172.21.15.16:0/4277577814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e000d650 con 0x7f21e8155550 2024-08-17T02:48:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.125+0000 7f21dea00640 1 -- 172.21.15.16:0/4277577814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e000da10 con 0x7f21e8155550 2024-08-17T02:48:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.125+0000 7f21ee000640 1 -- 172.21.15.16:0/4277577814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e81694b0 con 0x7f21e8155550 2024-08-17T02:48:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.125+0000 7f21ee000640 1 -- 172.21.15.16:0/4277577814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e8169a70 con 0x7f21e8155550 2024-08-17T02:48:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.126+0000 7f21ee000640 1 -- 172.21.15.16:0/4277577814 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e816c480 con 0x7f21e8155550 2024-08-17T02:48:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.127+0000 7f21dea00640 1 -- 172.21.15.16:0/4277577814 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f21e001e3b0 con 0x7f21e8155550 2024-08-17T02:48:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.128+0000 7f21dea00640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f21c8065150 0x7f21c8067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.128+0000 7f21dea00640 1 -- 172.21.15.16:0/4277577814 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f21e0082b50 con 0x7f21e8155550 2024-08-17T02:48:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.129+0000 7f21e7400640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f21c8065150 0x7f21c8067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.129+0000 7f21e7400640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f21c8065150 0x7f21c8067600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f21d8034000 tx=0x7f21d8032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:50.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.132+0000 7f21dea00640 1 -- 172.21.15.16:0/4277577814 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f21e816c480 con 0x7f21e8155550 2024-08-17T02:48:50.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.314+0000 7f21ee000640 1 -- 172.21.15.16:0/4277577814 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21e816c480 con 0x7f21c8065150 2024-08-17T02:48:50.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.316+0000 7f21dea00640 1 -- 172.21.15.16:0/4277577814 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50728 (secure 0 0 0) 0x7f21e816c480 con 0x7f21c8065150 2024-08-17T02:48:50.318 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-17T02:48:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.318+0000 7f21ee000640 1 -- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f21c8065150 msgr2=0x7f21c8067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.318+0000 7f21ee000640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f21c8065150 0x7f21c8067600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f21d8034000 tx=0x7f21d8032000 comp rx=0 tx=0).stop 2024-08-17T02:48:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.318+0000 7f21ee000640 1 -- 172.21.15.16:0/4277577814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8155550 msgr2=0x7f21e81538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.318+0000 7f21ee000640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8155550 0x7f21e81538c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f21e000cf60 tx=0x7f21e0007870 comp rx=0 tx=0).stop 2024-08-17T02:48:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.318+0000 7f21ee000640 1 -- 172.21.15.16:0/4277577814 shutdown_connections 2024-08-17T02:48:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.318+0000 7f21ee000640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f21c8065150 0x7f21c8067600 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.318+0000 7f21ee000640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21e8155550 0x7f21e81538c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.318+0000 7f21ee000640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f21e8155180 0x7f21e8152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.318+0000 7f21ee000640 1 --2- 172.21.15.16:0/4277577814 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f21e806e080 0x7f21e814c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.318+0000 7f21ee000640 1 -- 172.21.15.16:0/4277577814 >> 172.21.15.16:0/4277577814 conn(0x7f21e80cb9b0 msgr2=0x7f21e80c88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.318+0000 7f21ee000640 1 -- 172.21.15.16:0/4277577814 shutdown_connections 2024-08-17T02:48:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.318+0000 7f21ee000640 1 -- 172.21.15.16:0/4277577814 wait complete. 2024-08-17T02:48:50.335 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-08-17T02:48:50.291524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":2400908,"kb_used_data":5400,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752573812,"statfs":{"total":773094113280,"available":770635583488,"internally_reserved":0,"allocated":5529600,"data_stored":3020192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.804519"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-08-17T02:48:43.502604+0000","last_change":"2024-08-17T02:45:16.001117+0000","last_active":"2024-08-17T02:48:43.502604+0000","last_peered":"2024-08-17T02:48:43.502604+0000","last_clean":"2024-08-17T02:48:43.502604+0000","last_became_active":"2024-08-17T02:45:16.000788+0000","last_became_peered":"2024-08-17T02:45:16.000788+0000","last_unstale":"2024-08-17T02:48:43.502604+0000","last_undegraded":"2024-08-17T02:48:43.502604+0000","last_fullsized":"2024-08-17T02:48:43.502604+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_clean_scrub_stamp":"2024-08-17T02:45:14.986707+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:27:37.295374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.81799999999999995},"min":{"1min":0.61499999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90900000000000003,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.53100000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.88900000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67600000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.64200000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96399999999999997,"5min":2.202,"15min":2.202},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.65900000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.877,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.70699999999999996}]},{"osd":2,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81899999999999995,"15min":0.84699999999999998},"min":{"1min":0.57799999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94399999999999995,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.90900000000000003,"15min":0.95499999999999996},"min":{"1min":0.69799999999999995,"5min":0.621,"15min":0.621},"max":{"1min":0.93400000000000005,"5min":4.6870000000000003,"15min":4.6870000000000003},"last":0.77400000000000002}]},{"osd":3,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76100000000000001,"15min":0.78000000000000003},"min":{"1min":0.57699999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":2.798,"15min":2.798},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72799999999999998,"15min":0.73299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89500000000000002,"5min":1.081,"15min":1.081},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.67800000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73999999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.81100000000000005}]},{"osd":5,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.66000000000000003},"min":{"1min":0.504,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67500000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79000000000000004,"5min":1.337,"15min":1.337},"last":0.75700000000000001}]},{"osd":6,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84599999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79900000000000004,"5min":1.052,"15min":1.052},"last":0.95999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73899999999999999,"15min":0.70099999999999996},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98599999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":0.875},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71699999999999997,"15min":0.65900000000000003},"min":{"1min":0.60699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.81399999999999995}]},{"osd":1,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.58599999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":0.751},"min":{"1min":0.61799999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.998,"5min":5.0970000000000004,"15min":5.0970000000000004},"last":0.79900000000000004}]},{"osd":2,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66800000000000004,"15min":0.64800000000000002},"min":{"1min":0.51500000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75,"15min":0.69899999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":5.0579999999999998,"15min":5.0579999999999998},"last":0.76300000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.85099999999999998,"15min":0.84399999999999997},"min":{"1min":0.60799999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93100000000000005,"5min":5.1849999999999996,"15min":5.1849999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.76100000000000001},"min":{"1min":0.60699999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.052,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.83899999999999997}]},{"osd":4,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77700000000000002,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84399999999999997,"5min":5.7030000000000003,"15min":5.7030000000000003},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.47899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.82199999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.74399999999999999,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94999999999999996,"5min":5.1639999999999997,"15min":5.1639999999999997},"last":0.71099999999999997}]},{"osd":7,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.73999999999999999},"min":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91500000000000004,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.57399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.98899999999999999,"5min":1.405,"15min":1.405},"last":0.78200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1142784,"data_stored":822418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.47699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.58299999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.155,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.75}]},{"osd":1,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68400000000000005,"15min":0.65200000000000002},"min":{"1min":0.57799999999999996,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91100000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67800000000000005,"15min":0.64400000000000002},"min":{"1min":0.52100000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.57599999999999996}]},{"osd":2,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.67000000000000004,"15min":0.64600000000000002},"min":{"1min":0.59499999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93100000000000005,"5min":0.999,"15min":0.999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.60299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.68300000000000005}]},{"osd":3,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.68200000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.202,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.82499999999999996}]},{"osd":4,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.43099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.48499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.68500000000000005},"min":{"1min":0.48999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.876,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.76600000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.624},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64800000000000002,"15min":0.61799999999999999},"min":{"1min":0.34100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.86899999999999999,"5min":1.073,"15min":1.073},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65400000000000003,"15min":0.64100000000000001},"min":{"1min":0.53400000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.89100000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.624,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86599999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96499999999999997,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.872}]},{"osd":2,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.68700000000000006},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68100000000000005,"15min":0.67400000000000004},"min":{"1min":0.52000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.82899999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.90600000000000003}]},{"osd":3,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.55600000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81100000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73099999999999998,"15min":0.71999999999999997},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94299999999999995,"5min":1.147,"15min":1.147},"last":0.77800000000000002}]},{"osd":5,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67400000000000004,"15min":0.69299999999999995},"min":{"1min":0.40300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78300000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66900000000000004,"15min":0.67800000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84799999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84399999999999997}]},{"osd":6,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.45200000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67300000000000004,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78800000000000003,"5min":1.2,"15min":1.2},"last":0.93600000000000005}]},{"osd":7,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70699999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.504,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.61899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302036,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069804,"statfs":{"total":96636764160,"available":96327479296,"internally_reserved":0,"allocated":532480,"data_stored":226402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.48699999999999999,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.627,"5min":1.901,"15min":1.901},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59699999999999998,"15min":0.63300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":3.552,"15min":3.552},"last":0.32800000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.33200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.67200000000000004,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73499999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.48399999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.64700000000000002,"15min":0.69799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73599999999999999,"5min":3.8149999999999999,"15min":3.8149999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.625,"15min":0.63900000000000001},"min":{"1min":0.502,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73399999999999999,"5min":1.974,"15min":1.974},"last":0.42599999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.266,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.89600000000000002,"15min":0.94999999999999996},"min":{"1min":0.56999999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.206,"5min":4.3360000000000003,"15min":4.3360000000000003},"last":0.65300000000000002}]},{"osd":5,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.84699999999999998,"15min":0.874},"min":{"1min":0.60199999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0860000000000001,"5min":3.371,"15min":3.371},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.873,"15min":0.89900000000000002},"min":{"1min":0.59399999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1679999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.91000000000000003,"15min":0.96099999999999997},"min":{"1min":0.58299999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.3109999999999999,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.91700000000000004},"min":{"1min":0.72699999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2350000000000001,"5min":3.331,"15min":3.331},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.94099999999999995,"15min":0.98299999999999998},"min":{"1min":0.66500000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.105,"5min":3.931,"15min":3.931},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.95699999999999996,"15min":1.0029999999999999},"min":{"1min":0.67600000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4279999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.69399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302056,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069784,"statfs":{"total":96636764160,"available":96327458816,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.443,"15min":0.45900000000000002},"min":{"1min":0.314,"5min":0.255,"15min":0.255},"max":{"1min":0.58399999999999996,"5min":1.843,"15min":1.843},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52600000000000002,"15min":0.54800000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58599999999999997,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.73499999999999999,"5min":1.903,"15min":1.903},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.61499999999999999,"15min":0.66200000000000003},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.60299999999999998,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.72399999999999998,"15min":0.81899999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68799999999999994,"5min":7.5960000000000001,"15min":7.5960000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.77000000000000002,"15min":0.83999999999999997},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82299999999999995,"5min":6.0499999999999998,"15min":6.0499999999999998},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.85399999999999998},"min":{"1min":0.72899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.97599999999999998,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":1.0069999999999999,"15min":1.099},"min":{"1min":0.60999999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0860000000000001,"5min":5.7009999999999996,"15min":5.7009999999999996},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.96199999999999997,"15min":1.0569999999999999},"min":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.006,"5min":7.5220000000000002,"15min":7.5220000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.91200000000000003,"15min":0.97099999999999997},"min":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91900000000000004,"5min":5.968,"15min":5.968},"last":0.67200000000000004}]},{"osd":6,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":1.012,"15min":1.1100000000000001},"min":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99399999999999999,"5min":5.8780000000000001,"15min":5.8780000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.96999999999999997,"15min":1.0600000000000001},"min":{"1min":0.64500000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0009999999999999,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.75800000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":1.03,"15min":1.121},"min":{"1min":0.70699999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.054,"5min":5.9119999999999999,"15min":5.9119999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0189999999999999,"15min":1.113},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.95799999999999996,"5min":7.4800000000000004,"15min":7.4800000000000004},"last":0.70399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1126400,"data_stored":816770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.316,"5min":0.255,"15min":0.255},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.222},{"interface":"front","average":{"1min":0.503,"5min":0.57199999999999995,"15min":0.59999999999999998},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.70699999999999996,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.27200000000000002}]},{"osd":2,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69199999999999995,"5min":2.165,"15min":2.165},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.40100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.64500000000000002,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.38900000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.47899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82299999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.64100000000000001},"min":{"1min":0.41599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.85899999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.66400000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0229999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.66000000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.755}]},{"osd":5,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.622,"5min":0.501,"15min":0.501},"max":{"1min":1.034,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.68500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.085,"5min":1.131,"15min":1.131},"last":0.80000000000000004}]},{"osd":6,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.65000000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.1060000000000001,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":0.88900000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.85599999999999998,"15min":0.872},"min":{"1min":0.67000000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0700000000000001,"5min":2.577,"15min":2.577},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.63500000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.026,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.77200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302036,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069804,"statfs":{"total":96636764160,"available":96327479296,"internally_reserved":0,"allocated":532480,"data_stored":226402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.317,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.46899999999999997,"15min":0.46100000000000002},"min":{"1min":0.40000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67400000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.33500000000000002}]},{"osd":2,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70999999999999996,"5min":1.105,"15min":1.105},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.44600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.871,"15min":0.871},"last":0.51600000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.46100000000000002},"min":{"1min":0.39200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.754,"5min":1.181,"15min":1.181},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78400000000000003,"15min":0.77900000000000003},"min":{"1min":0.67600000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.093,"5min":1.468,"15min":1.468},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.71899999999999997,"15min":0.70799999999999996},"min":{"1min":0.56499999999999995,"5min":0.44,"15min":0.44},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.879}]},{"osd":5,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.59399999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75600000000000001,"15min":0.753},"min":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.91700000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.78500000000000003}]},{"osd":6,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96599999999999997,"5min":1.444,"15min":1.444},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80600000000000005,"15min":0.81499999999999995},"min":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.01,"5min":1.756,"15min":1.756},"last":0.90600000000000003}]},{"osd":7,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81899999999999995,"15min":0.81999999999999995},"min":{"1min":0.67800000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1339999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.87},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.65800000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99299999999999999,"5min":1.298,"15min":1.298},"last":0.95099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:48:50.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:48:50.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:48:50.388 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:48:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.489+0000 7f2eaa200640 1 Processor -- start 2024-08-17T02:48:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.489+0000 7f2eaa200640 1 -- start start 2024-08-17T02:48:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.490+0000 7f2eaa200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2ea40cc010 0x7f2ea40ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.490+0000 7f2eaa200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ea415c7c0 0x7f2ea415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.490+0000 7f2eaa200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea40ab950 con 0x7f2ea40a98e0 2024-08-17T02:48:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.490+0000 7f2eaa200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea40abc30 con 0x7f2ea415c7c0 2024-08-17T02:48:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.490+0000 7f2eaa200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea40abf10 con 0x7f2ea40cc010 2024-08-17T02:48:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.490+0000 7f2ea9800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ea415c7c0 0x7f2ea415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.490+0000 7f2ea9800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ea415c7c0 0x7f2ea415cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39632/0 (socket says 172.21.15.16:39632) 2024-08-17T02:48:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.490+0000 7f2ea9800640 1 -- 172.21.15.16:0/3169392734 learned_addr learned my addr 172.21.15.16:0/3169392734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:50.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.490+0000 7f2ea8e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f2ea40cc010 0x7f2ea40ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.491+0000 7f2ea9800640 1 -- 172.21.15.16:0/3169392734 >> v2:172.21.15.142:3300/0 conn(0x7f2ea40cc010 msgr2=0x7f2ea40ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.491+0000 7f2ea9800640 1 --2- 172.21.15.16:0/3169392734 >> v2:172.21.15.142:3300/0 conn(0x7f2ea40cc010 0x7f2ea40ab2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.491+0000 7f2ea9800640 1 -- 172.21.15.16:0/3169392734 >> v1:172.21.15.142:6789/0 conn(0x7f2ea40a98e0 legacy=0x7f2ea40afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.491+0000 7f2ea9800640 1 -- 172.21.15.16:0/3169392734 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea40af970 con 0x7f2ea415c7c0 2024-08-17T02:48:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.492+0000 7f2ea9800640 1 --2- 172.21.15.16:0/3169392734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea415c7c0 0x7f2ea415cc00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4064c30 tx=0x7f2e9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6642c4f1ba1c5c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.492+0000 7f2ea3400640 1 -- 172.21.15.16:0/3169392734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c031400 con 0x7f2ea415c7c0 2024-08-17T02:48:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.492+0000 7f2ea3400640 1 -- 172.21.15.16:0/3169392734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c02d050 con 0x7f2ea415c7c0 2024-08-17T02:48:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.492+0000 7f2ea3400640 1 -- 172.21.15.16:0/3169392734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0317d0 con 0x7f2ea415c7c0 2024-08-17T02:48:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.492+0000 7f2eaa200640 1 -- 172.21.15.16:0/3169392734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea415c7c0 msgr2=0x7f2ea415cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.492+0000 7f2eaa200640 1 --2- 172.21.15.16:0/3169392734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea415c7c0 0x7f2ea415cc00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4064c30 tx=0x7f2e9c009840 comp rx=0 tx=0).stop 2024-08-17T02:48:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.493+0000 7f2eaa200640 1 -- 172.21.15.16:0/3169392734 shutdown_connections 2024-08-17T02:48:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.493+0000 7f2eaa200640 1 --2- 172.21.15.16:0/3169392734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea415c7c0 0x7f2ea415cc00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.493+0000 7f2eaa200640 1 --2- 172.21.15.16:0/3169392734 >> v2:172.21.15.142:3300/0 conn(0x7f2ea40cc010 0x7f2ea40ab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.493+0000 7f2eaa200640 1 -- 172.21.15.16:0/3169392734 >> 172.21.15.16:0/3169392734 conn(0x7f2ea4067de0 msgr2=0x7f2ea40681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:50.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.493+0000 7f2eaa200640 1 -- 172.21.15.16:0/3169392734 shutdown_connections 2024-08-17T02:48:50.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.493+0000 7f2eaa200640 1 -- 172.21.15.16:0/3169392734 wait complete. 2024-08-17T02:48:50.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.494+0000 7f2eaa200640 1 Processor -- start 2024-08-17T02:48:50.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.494+0000 7f2eaa200640 1 -- start start 2024-08-17T02:48:50.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.495+0000 7f2eaa200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40a98e0 0x7f2ea41662e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.495+0000 7f2eaa200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ea40cc010 0x7f2ea415a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.495+0000 7f2eaa200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ea415c7c0 0x7f2ea415a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.495+0000 7f2eaa200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea41691d0 con 0x7f2ea40a98e0 2024-08-17T02:48:50.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.495+0000 7f2eaa200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea415b680 con 0x7f2ea415c7c0 2024-08-17T02:48:50.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.495+0000 7f2eaa200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea4166820 con 0x7f2ea40cc010 2024-08-17T02:48:50.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.495+0000 7f2ea8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40a98e0 0x7f2ea41662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.495+0000 7f2ea8e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40a98e0 0x7f2ea41662e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39640/0 (socket says 172.21.15.16:39640) 2024-08-17T02:48:50.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.495+0000 7f2ea8e00640 1 -- 172.21.15.16:0/716324857 learned_addr learned my addr 172.21.15.16:0/716324857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:50.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.495+0000 7f2ea9800640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ea415c7c0 0x7f2ea415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.495+0000 7f2ea3e00640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ea40cc010 0x7f2ea415a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.496+0000 7f2ea8e00640 1 -- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ea40cc010 msgr2=0x7f2ea415a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.496+0000 7f2ea8e00640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ea40cc010 0x7f2ea415a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.496+0000 7f2ea8e00640 1 -- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ea415c7c0 msgr2=0x7f2ea415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.496+0000 7f2ea8e00640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ea415c7c0 0x7f2ea415a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.496+0000 7f2ea8e00640 1 -- 172.21.15.16:0/716324857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c009ae0 con 0x7f2ea40a98e0 2024-08-17T02:48:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.496+0000 7f2ea9800640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ea415c7c0 0x7f2ea415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:48:50.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.496+0000 7f2ea8e00640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40a98e0 0x7f2ea41662e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800ea50 tx=0x7f2e980090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.496+0000 7f2e97e00640 1 -- 172.21.15.16:0/716324857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9800ca10 con 0x7f2ea40a98e0 2024-08-17T02:48:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.496+0000 7f2e97e00640 1 -- 172.21.15.16:0/716324857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9800cb70 con 0x7f2ea40a98e0 2024-08-17T02:48:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.496+0000 7f2e97e00640 1 -- 172.21.15.16:0/716324857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e98013470 con 0x7f2ea40a98e0 2024-08-17T02:48:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.496+0000 7f2eaa200640 1 -- 172.21.15.16:0/716324857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea415ade0 con 0x7f2ea40a98e0 2024-08-17T02:48:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.497+0000 7f2eaa200640 1 -- 172.21.15.16:0/716324857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea415b3a0 con 0x7f2ea40a98e0 2024-08-17T02:48:50.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.498+0000 7f2eaa200640 1 -- 172.21.15.16:0/716324857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea4071050 con 0x7f2ea40a98e0 2024-08-17T02:48:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.498+0000 7f2e97e00640 1 -- 172.21.15.16:0/716324857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f2e98013860 con 0x7f2ea40a98e0 2024-08-17T02:48:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.499+0000 7f2e97e00640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2e84065150 0x7f2e84067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.499+0000 7f2e97e00640 1 -- 172.21.15.16:0/716324857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f2e98082620 con 0x7f2ea40a98e0 2024-08-17T02:48:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.500+0000 7f2ea3e00640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2e84065150 0x7f2e84067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.500+0000 7f2ea3e00640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2e84065150 0x7f2e84067600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40aa110 tx=0x7f2e90007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:50.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.503+0000 7f2e97e00640 1 -- 172.21.15.16:0/716324857 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f2e98050a50 con 0x7f2ea40a98e0 2024-08-17T02:48:50.684 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.683+0000 7f2eaa200640 1 -- 172.21.15.16:0/716324857 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ea40b1620 con 0x7f2e84065150 2024-08-17T02:48:50.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.686+0000 7f2e97e00640 1 -- 172.21.15.16:0/716324857 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50728 (secure 0 0 0) 0x7f2ea40b1620 con 0x7f2e84065150 2024-08-17T02:48:50.687 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:50.688 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-17T02:48:50.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.688+0000 7f2eaa200640 1 -- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2e84065150 msgr2=0x7f2e84067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.688+0000 7f2eaa200640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2e84065150 0x7f2e84067600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40aa110 tx=0x7f2e90007e30 comp rx=0 tx=0).stop 2024-08-17T02:48:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.688+0000 7f2eaa200640 1 -- 172.21.15.16:0/716324857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40a98e0 msgr2=0x7f2ea41662e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.688+0000 7f2eaa200640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40a98e0 0x7f2ea41662e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800ea50 tx=0x7f2e980090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.688+0000 7f2eaa200640 1 -- 172.21.15.16:0/716324857 shutdown_connections 2024-08-17T02:48:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.688+0000 7f2eaa200640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f2e84065150 0x7f2e84067600 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.688+0000 7f2eaa200640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ea415c7c0 0x7f2ea415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.688+0000 7f2eaa200640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f2ea40cc010 0x7f2ea415a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.688+0000 7f2eaa200640 1 --2- 172.21.15.16:0/716324857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ea40a98e0 0x7f2ea41662e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.688+0000 7f2eaa200640 1 -- 172.21.15.16:0/716324857 >> 172.21.15.16:0/716324857 conn(0x7f2ea4067de0 msgr2=0x7f2ea40b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.688+0000 7f2eaa200640 1 -- 172.21.15.16:0/716324857 shutdown_connections 2024-08-17T02:48:50.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.688+0000 7f2eaa200640 1 -- 172.21.15.16:0/716324857 wait complete. 2024-08-17T02:48:50.704 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-08-17T02:48:50.291524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":2400908,"kb_used_data":5400,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752573812,"statfs":{"total":773094113280,"available":770635583488,"internally_reserved":0,"allocated":5529600,"data_stored":3020192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.804519"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-08-17T02:48:43.502604+0000","last_change":"2024-08-17T02:45:16.001117+0000","last_active":"2024-08-17T02:48:43.502604+0000","last_peered":"2024-08-17T02:48:43.502604+0000","last_clean":"2024-08-17T02:48:43.502604+0000","last_became_active":"2024-08-17T02:45:16.000788+0000","last_became_peered":"2024-08-17T02:45:16.000788+0000","last_unstale":"2024-08-17T02:48:43.502604+0000","last_undegraded":"2024-08-17T02:48:43.502604+0000","last_fullsized":"2024-08-17T02:48:43.502604+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_clean_scrub_stamp":"2024-08-17T02:45:14.986707+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:27:37.295374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.81799999999999995},"min":{"1min":0.61499999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90900000000000003,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.53100000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.88900000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67600000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.64200000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96399999999999997,"5min":2.202,"15min":2.202},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.65900000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.877,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.70699999999999996}]},{"osd":2,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81899999999999995,"15min":0.84699999999999998},"min":{"1min":0.57799999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94399999999999995,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.90900000000000003,"15min":0.95499999999999996},"min":{"1min":0.69799999999999995,"5min":0.621,"15min":0.621},"max":{"1min":0.93400000000000005,"5min":4.6870000000000003,"15min":4.6870000000000003},"last":0.77400000000000002}]},{"osd":3,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76100000000000001,"15min":0.78000000000000003},"min":{"1min":0.57699999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":2.798,"15min":2.798},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72799999999999998,"15min":0.73299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89500000000000002,"5min":1.081,"15min":1.081},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.67800000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73999999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.81100000000000005}]},{"osd":5,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.66000000000000003},"min":{"1min":0.504,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67500000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79000000000000004,"5min":1.337,"15min":1.337},"last":0.75700000000000001}]},{"osd":6,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84599999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79900000000000004,"5min":1.052,"15min":1.052},"last":0.95999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73899999999999999,"15min":0.70099999999999996},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98599999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":0.875},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71699999999999997,"15min":0.65900000000000003},"min":{"1min":0.60699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.81399999999999995}]},{"osd":1,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.58599999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":0.751},"min":{"1min":0.61799999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.998,"5min":5.0970000000000004,"15min":5.0970000000000004},"last":0.79900000000000004}]},{"osd":2,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66800000000000004,"15min":0.64800000000000002},"min":{"1min":0.51500000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75,"15min":0.69899999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":5.0579999999999998,"15min":5.0579999999999998},"last":0.76300000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.85099999999999998,"15min":0.84399999999999997},"min":{"1min":0.60799999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93100000000000005,"5min":5.1849999999999996,"15min":5.1849999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.76100000000000001},"min":{"1min":0.60699999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.052,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.83899999999999997}]},{"osd":4,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77700000000000002,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84399999999999997,"5min":5.7030000000000003,"15min":5.7030000000000003},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.47899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.82199999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.74399999999999999,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94999999999999996,"5min":5.1639999999999997,"15min":5.1639999999999997},"last":0.71099999999999997}]},{"osd":7,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.73999999999999999},"min":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91500000000000004,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.57399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.98899999999999999,"5min":1.405,"15min":1.405},"last":0.78200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1142784,"data_stored":822418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.47699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.58299999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.155,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.75}]},{"osd":1,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68400000000000005,"15min":0.65200000000000002},"min":{"1min":0.57799999999999996,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91100000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67800000000000005,"15min":0.64400000000000002},"min":{"1min":0.52100000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.57599999999999996}]},{"osd":2,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.67000000000000004,"15min":0.64600000000000002},"min":{"1min":0.59499999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93100000000000005,"5min":0.999,"15min":0.999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.60299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.68300000000000005}]},{"osd":3,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.68200000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.202,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.82499999999999996}]},{"osd":4,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.43099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.48499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.68500000000000005},"min":{"1min":0.48999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.876,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.76600000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.624},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64800000000000002,"15min":0.61799999999999999},"min":{"1min":0.34100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.86899999999999999,"5min":1.073,"15min":1.073},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65400000000000003,"15min":0.64100000000000001},"min":{"1min":0.53400000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.89100000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.624,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86599999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96499999999999997,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.872}]},{"osd":2,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.68700000000000006},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68100000000000005,"15min":0.67400000000000004},"min":{"1min":0.52000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.82899999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.90600000000000003}]},{"osd":3,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.55600000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81100000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73099999999999998,"15min":0.71999999999999997},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94299999999999995,"5min":1.147,"15min":1.147},"last":0.77800000000000002}]},{"osd":5,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67400000000000004,"15min":0.69299999999999995},"min":{"1min":0.40300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78300000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66900000000000004,"15min":0.67800000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84799999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84399999999999997}]},{"osd":6,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.45200000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67300000000000004,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78800000000000003,"5min":1.2,"15min":1.2},"last":0.93600000000000005}]},{"osd":7,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70699999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.504,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.61899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302036,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069804,"statfs":{"total":96636764160,"available":96327479296,"internally_reserved":0,"allocated":532480,"data_stored":226402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.48699999999999999,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.627,"5min":1.901,"15min":1.901},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59699999999999998,"15min":0.63300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":3.552,"15min":3.552},"last":0.32800000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.33200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.67200000000000004,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73499999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.48399999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.64700000000000002,"15min":0.69799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73599999999999999,"5min":3.8149999999999999,"15min":3.8149999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.625,"15min":0.63900000000000001},"min":{"1min":0.502,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73399999999999999,"5min":1.974,"15min":1.974},"last":0.42599999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.266,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.89600000000000002,"15min":0.94999999999999996},"min":{"1min":0.56999999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.206,"5min":4.3360000000000003,"15min":4.3360000000000003},"last":0.65300000000000002}]},{"osd":5,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.84699999999999998,"15min":0.874},"min":{"1min":0.60199999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0860000000000001,"5min":3.371,"15min":3.371},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.873,"15min":0.89900000000000002},"min":{"1min":0.59399999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1679999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.91000000000000003,"15min":0.96099999999999997},"min":{"1min":0.58299999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.3109999999999999,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.91700000000000004},"min":{"1min":0.72699999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2350000000000001,"5min":3.331,"15min":3.331},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.94099999999999995,"15min":0.98299999999999998},"min":{"1min":0.66500000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.105,"5min":3.931,"15min":3.931},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.95699999999999996,"15min":1.0029999999999999},"min":{"1min":0.67600000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4279999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.69399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302056,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069784,"statfs":{"total":96636764160,"available":96327458816,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.443,"15min":0.45900000000000002},"min":{"1min":0.314,"5min":0.255,"15min":0.255},"max":{"1min":0.58399999999999996,"5min":1.843,"15min":1.843},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52600000000000002,"15min":0.54800000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58599999999999997,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.73499999999999999,"5min":1.903,"15min":1.903},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.61499999999999999,"15min":0.66200000000000003},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.60299999999999998,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.72399999999999998,"15min":0.81899999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68799999999999994,"5min":7.5960000000000001,"15min":7.5960000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.77000000000000002,"15min":0.83999999999999997},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82299999999999995,"5min":6.0499999999999998,"15min":6.0499999999999998},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.85399999999999998},"min":{"1min":0.72899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.97599999999999998,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":1.0069999999999999,"15min":1.099},"min":{"1min":0.60999999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0860000000000001,"5min":5.7009999999999996,"15min":5.7009999999999996},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.96199999999999997,"15min":1.0569999999999999},"min":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.006,"5min":7.5220000000000002,"15min":7.5220000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.91200000000000003,"15min":0.97099999999999997},"min":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91900000000000004,"5min":5.968,"15min":5.968},"last":0.67200000000000004}]},{"osd":6,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":1.012,"15min":1.1100000000000001},"min":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99399999999999999,"5min":5.8780000000000001,"15min":5.8780000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.96999999999999997,"15min":1.0600000000000001},"min":{"1min":0.64500000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0009999999999999,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.75800000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":1.03,"15min":1.121},"min":{"1min":0.70699999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.054,"5min":5.9119999999999999,"15min":5.9119999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0189999999999999,"15min":1.113},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.95799999999999996,"5min":7.4800000000000004,"15min":7.4800000000000004},"last":0.70399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1126400,"data_stored":816770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.316,"5min":0.255,"15min":0.255},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.222},{"interface":"front","average":{"1min":0.503,"5min":0.57199999999999995,"15min":0.59999999999999998},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.70699999999999996,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.27200000000000002}]},{"osd":2,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69199999999999995,"5min":2.165,"15min":2.165},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.40100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.64500000000000002,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.38900000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.47899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82299999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.64100000000000001},"min":{"1min":0.41599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.85899999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.66400000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0229999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.66000000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.755}]},{"osd":5,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.622,"5min":0.501,"15min":0.501},"max":{"1min":1.034,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.68500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.085,"5min":1.131,"15min":1.131},"last":0.80000000000000004}]},{"osd":6,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.65000000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.1060000000000001,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":0.88900000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.85599999999999998,"15min":0.872},"min":{"1min":0.67000000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0700000000000001,"5min":2.577,"15min":2.577},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.63500000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.026,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.77200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302036,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069804,"statfs":{"total":96636764160,"available":96327479296,"internally_reserved":0,"allocated":532480,"data_stored":226402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.317,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.46899999999999997,"15min":0.46100000000000002},"min":{"1min":0.40000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67400000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.33500000000000002}]},{"osd":2,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70999999999999996,"5min":1.105,"15min":1.105},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.44600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.871,"15min":0.871},"last":0.51600000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.46100000000000002},"min":{"1min":0.39200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.754,"5min":1.181,"15min":1.181},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78400000000000003,"15min":0.77900000000000003},"min":{"1min":0.67600000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.093,"5min":1.468,"15min":1.468},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.71899999999999997,"15min":0.70799999999999996},"min":{"1min":0.56499999999999995,"5min":0.44,"15min":0.44},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.879}]},{"osd":5,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.59399999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75600000000000001,"15min":0.753},"min":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.91700000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.78500000000000003}]},{"osd":6,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96599999999999997,"5min":1.444,"15min":1.444},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80600000000000005,"15min":0.81499999999999995},"min":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.01,"5min":1.756,"15min":1.756},"last":0.90600000000000003}]},{"osd":7,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81899999999999995,"15min":0.81999999999999995},"min":{"1min":0.67800000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1339999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.87},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.65800000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99299999999999999,"5min":1.298,"15min":1.298},"last":0.95099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:48:50.705 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-08-17T02:48:50.705 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:48:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.894+0000 7f4ec7200640 1 Processor -- start 2024-08-17T02:48:50.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.894+0000 7f4ec7200640 1 -- start start 2024-08-17T02:48:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.895+0000 7f4ec7200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ec0155a20 0x7f4ec00adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.895+0000 7f4ec7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec006df00 0x7f4ec00ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.895+0000 7f4ec7200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec0156670 con 0x7f4ec0155df0 2024-08-17T02:48:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.895+0000 7f4ec7200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec00b0350 con 0x7f4ec0155a20 2024-08-17T02:48:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.895+0000 7f4ec7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec006e5e0 con 0x7f4ec006df00 2024-08-17T02:48:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.896+0000 7f4ebfe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ec0155a20 0x7f4ec00adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.896+0000 7f4ebfe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ec0155a20 0x7f4ec00adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39654/0 (socket says 172.21.15.16:39654) 2024-08-17T02:48:50.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.896+0000 7f4ebfe00640 1 -- 172.21.15.16:0/1031219992 learned_addr learned my addr 172.21.15.16:0/1031219992 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:50.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.896+0000 7f4ec5200640 1 --2- 172.21.15.16:0/1031219992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec006df00 0x7f4ec00ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.896+0000 7f4ebfe00640 1 -- 172.21.15.16:0/1031219992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec006df00 msgr2=0x7f4ec00ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.896+0000 7f4ebfe00640 1 --2- 172.21.15.16:0/1031219992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec006df00 0x7f4ec00ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.896+0000 7f4ebfe00640 1 -- 172.21.15.16:0/1031219992 >> v1:172.21.15.142:6789/0 conn(0x7f4ec0155df0 legacy=0x7f4ec006d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.896+0000 7f4ebfe00640 1 -- 172.21.15.16:0/1031219992 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec006e8c0 con 0x7f4ec0155a20 2024-08-17T02:48:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.896+0000 7f4ec5200640 1 --2- 172.21.15.16:0/1031219992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec006df00 0x7f4ec00ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.897+0000 7f4ebfe00640 1 --2- 172.21.15.16:0/1031219992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec0155a20 0x7f4ec00adf10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009910 tx=0x7f4eb00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af959fd05212983 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.897+0000 7f4ebea00640 1 -- 172.21.15.16:0/1031219992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb002ae40 con 0x7f4ec0155a20 2024-08-17T02:48:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.897+0000 7f4ebea00640 1 -- 172.21.15.16:0/1031219992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb0032440 con 0x7f4ec0155a20 2024-08-17T02:48:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.897+0000 7f4ebea00640 1 -- 172.21.15.16:0/1031219992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0032820 con 0x7f4ec0155a20 2024-08-17T02:48:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.898+0000 7f4ec7200640 1 -- 172.21.15.16:0/1031219992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec0155a20 msgr2=0x7f4ec00adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.898+0000 7f4ec7200640 1 --2- 172.21.15.16:0/1031219992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec0155a20 0x7f4ec00adf10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009910 tx=0x7f4eb00091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.898+0000 7f4ec7200640 1 -- 172.21.15.16:0/1031219992 shutdown_connections 2024-08-17T02:48:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.898+0000 7f4ec7200640 1 --2- 172.21.15.16:0/1031219992 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec006df00 0x7f4ec00ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.898+0000 7f4ec7200640 1 --2- 172.21.15.16:0/1031219992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec0155a20 0x7f4ec00adf10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.898+0000 7f4ec7200640 1 -- 172.21.15.16:0/1031219992 >> 172.21.15.16:0/1031219992 conn(0x7f4ec00cb9b0 msgr2=0x7f4ec00cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.898+0000 7f4ec7200640 1 -- 172.21.15.16:0/1031219992 shutdown_connections 2024-08-17T02:48:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.898+0000 7f4ec7200640 1 -- 172.21.15.16:0/1031219992 wait complete. 2024-08-17T02:48:50.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.899+0000 7f4ec7200640 1 Processor -- start 2024-08-17T02:48:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.900+0000 7f4ec7200640 1 -- start start 2024-08-17T02:48:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.900+0000 7f4ec7200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec006df00 0x7f4ec0169110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.900+0000 7f4ec7200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec0155a20 0x7f4ec00ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.900+0000 7f4ec7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec0155df0 0x7f4ec014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.900+0000 7f4ec7200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0152af0 con 0x7f4ec006df00 2024-08-17T02:48:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.900+0000 7f4ec7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec00acfd0 con 0x7f4ec0155df0 2024-08-17T02:48:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.900+0000 7f4ec7200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec00ac3c0 con 0x7f4ec0155a20 2024-08-17T02:48:50.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.901+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec006df00 0x7f4ec0169110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.901+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec006df00 0x7f4ec0169110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39668/0 (socket says 172.21.15.16:39668) 2024-08-17T02:48:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.901+0000 7f4ebfe00640 1 -- 172.21.15.16:0/3764681367 learned_addr learned my addr 172.21.15.16:0/3764681367 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.901+0000 7f4ebf400640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec0155a20 0x7f4ec00ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.901+0000 7f4ec5200640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec0155df0 0x7f4ec014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.901+0000 7f4ebfe00640 1 -- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec0155a20 msgr2=0x7f4ec00ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.901+0000 7f4ebfe00640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec0155a20 0x7f4ec00ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.901+0000 7f4ebfe00640 1 -- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec0155df0 msgr2=0x7f4ec014fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.901+0000 7f4ebfe00640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec0155df0 0x7f4ec014fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.901+0000 7f4ebfe00640 1 -- 172.21.15.16:0/3764681367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb0009450 con 0x7f4ec006df00 2024-08-17T02:48:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.901+0000 7f4ec5200640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec0155df0 0x7f4ec014fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.902+0000 7f4ebfe00640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec006df00 0x7f4ec0169110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0035040 tx=0x7f4eb0032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.902+0000 7f4eb7400640 1 -- 172.21.15.16:0/3764681367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0000c90 con 0x7f4ec006df00 2024-08-17T02:48:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.902+0000 7f4eb7400640 1 -- 172.21.15.16:0/3764681367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb0031a40 con 0x7f4ec006df00 2024-08-17T02:48:50.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.902+0000 7f4eb7400640 1 -- 172.21.15.16:0/3764681367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0031e00 con 0x7f4ec006df00 2024-08-17T02:48:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.902+0000 7f4ec7200640 1 -- 172.21.15.16:0/3764681367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec00ac6a0 con 0x7f4ec006df00 2024-08-17T02:48:50.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.902+0000 7f4ec7200640 1 -- 172.21.15.16:0/3764681367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec00acc60 con 0x7f4ec006df00 2024-08-17T02:48:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.903+0000 7f4ec7200640 1 -- 172.21.15.16:0/3764681367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec0064db0 con 0x7f4ec006df00 2024-08-17T02:48:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.904+0000 7f4eb7400640 1 -- 172.21.15.16:0/3764681367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f4eb0039040 con 0x7f4ec006df00 2024-08-17T02:48:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.905+0000 7f4eb7400640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4ea0065150 0x7f4ea0067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.905+0000 7f4eb7400640 1 -- 172.21.15.16:0/3764681367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f4eb002ae40 con 0x7f4ec006df00 2024-08-17T02:48:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.905+0000 7f4ebf400640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4ea0065150 0x7f4ea0067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:50.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.906+0000 7f4ebf400640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4ea0065150 0x7f4ea0067600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00cac10 tx=0x7f4eac007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:50.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:50.908+0000 7f4eb7400640 1 -- 172.21.15.16:0/3764681367 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f4eb0070080 con 0x7f4ec006df00 2024-08-17T02:48:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.088+0000 7f4ec7200640 1 -- 172.21.15.16:0/3764681367 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ec006aac0 con 0x7f4ea0065150 2024-08-17T02:48:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.090+0000 7f4eb7400640 1 -- 172.21.15.16:0/3764681367 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50728 (secure 0 0 0) 0x7f4ec006aac0 con 0x7f4ea0065150 2024-08-17T02:48:51.092 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-17T02:48:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.093+0000 7f4ec7200640 1 -- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4ea0065150 msgr2=0x7f4ea0067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.093+0000 7f4ec7200640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4ea0065150 0x7f4ea0067600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00cac10 tx=0x7f4eac007e30 comp rx=0 tx=0).stop 2024-08-17T02:48:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.093+0000 7f4ec7200640 1 -- 172.21.15.16:0/3764681367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec006df00 msgr2=0x7f4ec0169110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.093+0000 7f4ec7200640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec006df00 0x7f4ec0169110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0035040 tx=0x7f4eb0032000 comp rx=0 tx=0).stop 2024-08-17T02:48:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.093+0000 7f4ec7200640 1 -- 172.21.15.16:0/3764681367 shutdown_connections 2024-08-17T02:48:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.093+0000 7f4ec7200640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4ea0065150 0x7f4ea0067600 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.093+0000 7f4ec7200640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ec0155df0 0x7f4ec014fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.093+0000 7f4ec7200640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f4ec0155a20 0x7f4ec00ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.093+0000 7f4ec7200640 1 --2- 172.21.15.16:0/3764681367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec006df00 0x7f4ec0169110 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.093+0000 7f4ec7200640 1 -- 172.21.15.16:0/3764681367 >> 172.21.15.16:0/3764681367 conn(0x7f4ec00cb9b0 msgr2=0x7f4ec0158240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.093+0000 7f4ec7200640 1 -- 172.21.15.16:0/3764681367 shutdown_connections 2024-08-17T02:48:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.093+0000 7f4ec7200640 1 -- 172.21.15.16:0/3764681367 wait complete. 2024-08-17T02:48:51.110 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-08-17T02:48:50.291524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":2400908,"kb_used_data":5400,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752573812,"statfs":{"total":773094113280,"available":770635583488,"internally_reserved":0,"allocated":5529600,"data_stored":3020192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2338,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-45,"num_write_kb":-13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.804519"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-08-17T02:48:43.502604+0000","last_change":"2024-08-17T02:45:16.001117+0000","last_active":"2024-08-17T02:48:43.502604+0000","last_peered":"2024-08-17T02:48:43.502604+0000","last_clean":"2024-08-17T02:48:43.502604+0000","last_became_active":"2024-08-17T02:45:16.000788+0000","last_became_peered":"2024-08-17T02:45:16.000788+0000","last_unstale":"2024-08-17T02:48:43.502604+0000","last_undegraded":"2024-08-17T02:48:43.502604+0000","last_fullsized":"2024-08-17T02:48:43.502604+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_clean_scrub_stamp":"2024-08-17T02:45:14.986707+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:27:37.295374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.81799999999999995},"min":{"1min":0.61499999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90900000000000003,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.53100000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.88900000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67600000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.64200000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96399999999999997,"5min":2.202,"15min":2.202},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.65900000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.877,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.70699999999999996}]},{"osd":2,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81899999999999995,"15min":0.84699999999999998},"min":{"1min":0.57799999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94399999999999995,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.90900000000000003,"15min":0.95499999999999996},"min":{"1min":0.69799999999999995,"5min":0.621,"15min":0.621},"max":{"1min":0.93400000000000005,"5min":4.6870000000000003,"15min":4.6870000000000003},"last":0.77400000000000002}]},{"osd":3,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76100000000000001,"15min":0.78000000000000003},"min":{"1min":0.57699999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":2.798,"15min":2.798},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72799999999999998,"15min":0.73299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89500000000000002,"5min":1.081,"15min":1.081},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.67800000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73999999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.81100000000000005}]},{"osd":5,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.66000000000000003},"min":{"1min":0.504,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67500000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79000000000000004,"5min":1.337,"15min":1.337},"last":0.75700000000000001}]},{"osd":6,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84599999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79900000000000004,"5min":1.052,"15min":1.052},"last":0.95999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73899999999999999,"15min":0.70099999999999996},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98599999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":0.875},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71699999999999997,"15min":0.65900000000000003},"min":{"1min":0.60699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.81399999999999995}]},{"osd":1,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.58599999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":0.751},"min":{"1min":0.61799999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.998,"5min":5.0970000000000004,"15min":5.0970000000000004},"last":0.79900000000000004}]},{"osd":2,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66800000000000004,"15min":0.64800000000000002},"min":{"1min":0.51500000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75,"15min":0.69899999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":5.0579999999999998,"15min":5.0579999999999998},"last":0.76300000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.85099999999999998,"15min":0.84399999999999997},"min":{"1min":0.60799999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93100000000000005,"5min":5.1849999999999996,"15min":5.1849999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.76100000000000001},"min":{"1min":0.60699999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.052,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.83899999999999997}]},{"osd":4,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77700000000000002,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84399999999999997,"5min":5.7030000000000003,"15min":5.7030000000000003},"last":0.65200000000000002}]},{"osd":5,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.47899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.82199999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.74399999999999999,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94999999999999996,"5min":5.1639999999999997,"15min":5.1639999999999997},"last":0.71099999999999997}]},{"osd":7,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.73999999999999999},"min":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91500000000000004,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.57399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.98899999999999999,"5min":1.405,"15min":1.405},"last":0.78200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1142784,"data_stored":822418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.47699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.58299999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.155,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.75}]},{"osd":1,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68400000000000005,"15min":0.65200000000000002},"min":{"1min":0.57799999999999996,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91100000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67800000000000005,"15min":0.64400000000000002},"min":{"1min":0.52100000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.57599999999999996}]},{"osd":2,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.67000000000000004,"15min":0.64600000000000002},"min":{"1min":0.59499999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93100000000000005,"5min":0.999,"15min":0.999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.60299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.68300000000000005}]},{"osd":3,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.68200000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.202,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.82499999999999996}]},{"osd":4,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.43099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.48499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.68500000000000005},"min":{"1min":0.48999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.876,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.76600000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.624},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64800000000000002,"15min":0.61799999999999999},"min":{"1min":0.34100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.86899999999999999,"5min":1.073,"15min":1.073},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65400000000000003,"15min":0.64100000000000001},"min":{"1min":0.53400000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.89100000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.624,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86599999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96499999999999997,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.872}]},{"osd":2,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.68700000000000006},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68100000000000005,"15min":0.67400000000000004},"min":{"1min":0.52000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.82899999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.90600000000000003}]},{"osd":3,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.55600000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81100000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73099999999999998,"15min":0.71999999999999997},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94299999999999995,"5min":1.147,"15min":1.147},"last":0.77800000000000002}]},{"osd":5,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67400000000000004,"15min":0.69299999999999995},"min":{"1min":0.40300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78300000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66900000000000004,"15min":0.67800000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84799999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.84399999999999997}]},{"osd":6,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.45200000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67300000000000004,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78800000000000003,"5min":1.2,"15min":1.2},"last":0.93600000000000005}]},{"osd":7,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70699999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.504,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.61899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302036,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069804,"statfs":{"total":96636764160,"available":96327479296,"internally_reserved":0,"allocated":532480,"data_stored":226402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.48699999999999999,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.627,"5min":1.901,"15min":1.901},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59699999999999998,"15min":0.63300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":3.552,"15min":3.552},"last":0.32800000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.33200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.67200000000000004,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73499999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.48399999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.64700000000000002,"15min":0.69799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73599999999999999,"5min":3.8149999999999999,"15min":3.8149999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.625,"15min":0.63900000000000001},"min":{"1min":0.502,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73399999999999999,"5min":1.974,"15min":1.974},"last":0.42599999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.266,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.89600000000000002,"15min":0.94999999999999996},"min":{"1min":0.56999999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.206,"5min":4.3360000000000003,"15min":4.3360000000000003},"last":0.65300000000000002}]},{"osd":5,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.84699999999999998,"15min":0.874},"min":{"1min":0.60199999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0860000000000001,"5min":3.371,"15min":3.371},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.873,"15min":0.89900000000000002},"min":{"1min":0.59399999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1679999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.91000000000000003,"15min":0.96099999999999997},"min":{"1min":0.58299999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.3109999999999999,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.91700000000000004},"min":{"1min":0.72699999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2350000000000001,"5min":3.331,"15min":3.331},"last":0.71199999999999997}]},{"osd":7,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.94099999999999995,"15min":0.98299999999999998},"min":{"1min":0.66500000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.105,"5min":3.931,"15min":3.931},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.95699999999999996,"15min":1.0029999999999999},"min":{"1min":0.67600000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4279999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.69399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302056,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069784,"statfs":{"total":96636764160,"available":96327458816,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.443,"15min":0.45900000000000002},"min":{"1min":0.314,"5min":0.255,"15min":0.255},"max":{"1min":0.58399999999999996,"5min":1.843,"15min":1.843},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52600000000000002,"15min":0.54800000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58599999999999997,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.40999999999999998}]},{"osd":1,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.73499999999999999,"5min":1.903,"15min":1.903},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.61499999999999999,"15min":0.66200000000000003},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.60299999999999998,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.45000000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.72399999999999998,"15min":0.81899999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68799999999999994,"5min":7.5960000000000001,"15min":7.5960000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.77000000000000002,"15min":0.83999999999999997},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82299999999999995,"5min":6.0499999999999998,"15min":6.0499999999999998},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.85399999999999998},"min":{"1min":0.72899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.97599999999999998,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":1.0069999999999999,"15min":1.099},"min":{"1min":0.60999999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0860000000000001,"5min":5.7009999999999996,"15min":5.7009999999999996},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.96199999999999997,"15min":1.0569999999999999},"min":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.006,"5min":7.5220000000000002,"15min":7.5220000000000002},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.91200000000000003,"15min":0.97099999999999997},"min":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91900000000000004,"5min":5.968,"15min":5.968},"last":0.67200000000000004}]},{"osd":6,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":1.012,"15min":1.1100000000000001},"min":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99399999999999999,"5min":5.8780000000000001,"15min":5.8780000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.96999999999999997,"15min":1.0600000000000001},"min":{"1min":0.64500000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0009999999999999,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.75800000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":1.03,"15min":1.121},"min":{"1min":0.70699999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.054,"5min":5.9119999999999999,"15min":5.9119999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0189999999999999,"15min":1.113},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.95799999999999996,"5min":7.4800000000000004,"15min":7.4800000000000004},"last":0.70399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302616,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069224,"statfs":{"total":96636764160,"available":96326885376,"internally_reserved":0,"allocated":1126400,"data_stored":816770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.316,"5min":0.255,"15min":0.255},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.222},{"interface":"front","average":{"1min":0.503,"5min":0.57199999999999995,"15min":0.59999999999999998},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.70699999999999996,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.27200000000000002}]},{"osd":2,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69199999999999995,"5min":2.165,"15min":2.165},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.40100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.64500000000000002,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.38900000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.47899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82299999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.64100000000000001},"min":{"1min":0.41599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.85899999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.40999999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.66400000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0229999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.66000000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.755}]},{"osd":5,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.622,"5min":0.501,"15min":0.501},"max":{"1min":1.034,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.68500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.085,"5min":1.131,"15min":1.131},"last":0.80000000000000004}]},{"osd":6,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.65000000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.1060000000000001,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":0.88900000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.85599999999999998,"15min":0.872},"min":{"1min":0.67000000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0700000000000001,"5min":2.577,"15min":2.577},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.63500000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.026,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.77200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302036,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069804,"statfs":{"total":96636764160,"available":96327479296,"internally_reserved":0,"allocated":532480,"data_stored":226402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.317,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.46899999999999997,"15min":0.46100000000000002},"min":{"1min":0.40000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67400000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.33500000000000002}]},{"osd":2,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70999999999999996,"5min":1.105,"15min":1.105},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.44600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.871,"15min":0.871},"last":0.51600000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.46100000000000002},"min":{"1min":0.39200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.754,"5min":1.181,"15min":1.181},"last":0.58499999999999996}]},{"osd":4,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78400000000000003,"15min":0.77900000000000003},"min":{"1min":0.67600000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.093,"5min":1.468,"15min":1.468},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.71899999999999997,"15min":0.70799999999999996},"min":{"1min":0.56499999999999995,"5min":0.44,"15min":0.44},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.879}]},{"osd":5,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.59399999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75600000000000001,"15min":0.753},"min":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.91700000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.78500000000000003}]},{"osd":6,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96599999999999997,"5min":1.444,"15min":1.444},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80600000000000005,"15min":0.81499999999999995},"min":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.01,"5min":1.756,"15min":1.756},"last":0.90600000000000003}]},{"osd":7,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81899999999999995,"15min":0.81999999999999995},"min":{"1min":0.67800000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1339999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.87},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.65800000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99299999999999999,"5min":1.298,"15min":1.298},"last":0.95099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:48:51.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-17T02:48:51.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.261+0000 7fa293600640 1 Processor -- start 2024-08-17T02:48:51.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.262+0000 7fa293600640 1 -- start start 2024-08-17T02:48:51.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.263+0000 7fa293600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28c155a20 0x7fa28c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:51.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.263+0000 7fa293600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa28c06df00 0x7fa28c0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:51.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.263+0000 7fa293600640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa28c156670 con 0x7fa28c155df0 2024-08-17T02:48:51.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.263+0000 7fa293600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28c0b0350 con 0x7fa28c06df00 2024-08-17T02:48:51.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.263+0000 7fa293600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c06e5e0 con 0x7fa28c155a20 2024-08-17T02:48:51.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.263+0000 7fa291600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa28c06df00 0x7fa28c0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:51.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.263+0000 7fa291600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa28c06df00 0x7fa28c0ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39682/0 (socket says 172.21.15.16:39682) 2024-08-17T02:48:51.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.263+0000 7fa291600640 1 -- 172.21.15.16:0/2384422799 learned_addr learned my addr 172.21.15.16:0/2384422799 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:51.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.263+0000 7fa290c00640 1 --2- 172.21.15.16:0/2384422799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28c155a20 0x7fa28c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:51.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.264+0000 7fa291600640 1 -- 172.21.15.16:0/2384422799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28c155a20 msgr2=0x7fa28c0adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:51.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.264+0000 7fa291600640 1 --2- 172.21.15.16:0/2384422799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28c155a20 0x7fa28c0adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.264+0000 7fa291600640 1 -- 172.21.15.16:0/2384422799 >> v1:172.21.15.142:6789/0 conn(0x7fa28c155df0 legacy=0x7fa28c06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:51.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.264+0000 7fa291600640 1 -- 172.21.15.16:0/2384422799 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c06e8c0 con 0x7fa28c06df00 2024-08-17T02:48:51.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.265+0000 7fa291600640 1 --2- 172.21.15.16:0/2384422799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c06df00 0x7fa28c0ad9b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa284009f20 tx=0x7fa284009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1eb4298113accb server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:51.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.265+0000 7fa28be00640 1 -- 172.21.15.16:0/2384422799 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284031400 con 0x7fa28c06df00 2024-08-17T02:48:51.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.265+0000 7fa28be00640 1 -- 172.21.15.16:0/2384422799 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa28402d050 con 0x7fa28c06df00 2024-08-17T02:48:51.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.265+0000 7fa28be00640 1 -- 172.21.15.16:0/2384422799 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840317d0 con 0x7fa28c06df00 2024-08-17T02:48:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.265+0000 7fa293600640 1 -- 172.21.15.16:0/2384422799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c06df00 msgr2=0x7fa28c0ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.265+0000 7fa293600640 1 --2- 172.21.15.16:0/2384422799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c06df00 0x7fa28c0ad9b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa284009f20 tx=0x7fa284009840 comp rx=0 tx=0).stop 2024-08-17T02:48:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.266+0000 7fa293600640 1 -- 172.21.15.16:0/2384422799 shutdown_connections 2024-08-17T02:48:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.266+0000 7fa293600640 1 --2- 172.21.15.16:0/2384422799 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c06df00 0x7fa28c0ad9b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.266+0000 7fa293600640 1 --2- 172.21.15.16:0/2384422799 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28c155a20 0x7fa28c0adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.266+0000 7fa293600640 1 -- 172.21.15.16:0/2384422799 >> 172.21.15.16:0/2384422799 conn(0x7fa28c0cb9b0 msgr2=0x7fa28c0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.266+0000 7fa293600640 1 -- 172.21.15.16:0/2384422799 shutdown_connections 2024-08-17T02:48:51.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.266+0000 7fa293600640 1 -- 172.21.15.16:0/2384422799 wait complete. 2024-08-17T02:48:51.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.267+0000 7fa293600640 1 Processor -- start 2024-08-17T02:48:51.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.267+0000 7fa293600640 1 -- start start 2024-08-17T02:48:51.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.268+0000 7fa293600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa28c06df00 0x7fa28c152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:51.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.268+0000 7fa293600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c155a20 0x7fa28c146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:51.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.268+0000 7fa293600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28c155df0 0x7fa28c147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:51.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.268+0000 7fa293600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c168f50 con 0x7fa28c155a20 2024-08-17T02:48:51.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.268+0000 7fa293600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c16d210 con 0x7fa28c06df00 2024-08-17T02:48:51.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.268+0000 7fa293600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c169160 con 0x7fa28c155df0 2024-08-17T02:48:51.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.268+0000 7fa283e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c155a20 0x7fa28c146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:51.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.268+0000 7fa283e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c155a20 0x7fa28c146ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39696/0 (socket says 172.21.15.16:39696) 2024-08-17T02:48:51.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.268+0000 7fa283e00640 1 -- 172.21.15.16:0/2047621631 learned_addr learned my addr 172.21.15.16:0/2047621631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:51.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.268+0000 7fa291600640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28c155df0 0x7fa28c147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:51.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.268+0000 7fa290c00640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa28c06df00 0x7fa28c152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:51.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.269+0000 7fa283e00640 1 -- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28c155df0 msgr2=0x7fa28c147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:51.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.269+0000 7fa283e00640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28c155df0 0x7fa28c147530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.269+0000 7fa283e00640 1 -- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa28c06df00 msgr2=0x7fa28c152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:51.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.269+0000 7fa283e00640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa28c06df00 0x7fa28c152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.269+0000 7fa283e00640 1 -- 172.21.15.16:0/2047621631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa284009ae0 con 0x7fa28c155a20 2024-08-17T02:48:51.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.269+0000 7fa283e00640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c155a20 0x7fa28c146ff0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa27400c4c0 tx=0x7fa274007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:51.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.269+0000 7fa288c00640 1 -- 172.21.15.16:0/2047621631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274011590 con 0x7fa28c155a20 2024-08-17T02:48:51.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.269+0000 7fa288c00640 1 -- 172.21.15.16:0/2047621631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa274011710 con 0x7fa28c155a20 2024-08-17T02:48:51.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.269+0000 7fa288c00640 1 -- 172.21.15.16:0/2047621631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274011ad0 con 0x7fa28c155a20 2024-08-17T02:48:51.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.269+0000 7fa293600640 1 -- 172.21.15.16:0/2047621631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c169440 con 0x7fa28c155a20 2024-08-17T02:48:51.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.269+0000 7fa293600640 1 -- 172.21.15.16:0/2047621631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c152b80 con 0x7fa28c155a20 2024-08-17T02:48:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.271+0000 7fa293600640 1 -- 172.21.15.16:0/2047621631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c169940 con 0x7fa28c155a20 2024-08-17T02:48:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.271+0000 7fa288c00640 1 -- 172.21.15.16:0/2047621631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fa27401b3b0 con 0x7fa28c155a20 2024-08-17T02:48:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.272+0000 7fa288c00640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa268065150 0x7fa268067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.273+0000 7fa288c00640 1 -- 172.21.15.16:0/2047621631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fa274081660 con 0x7fa28c155a20 2024-08-17T02:48:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.273+0000 7fa290c00640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa268065150 0x7fa268067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:51.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.274+0000 7fa290c00640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa268065150 0x7fa268067600 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009d20 tx=0x7fa27c0090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:51.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.276+0000 7fa288c00640 1 -- 172.21.15.16:0/2047621631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fa27404f9d0 con 0x7fa28c155a20 2024-08-17T02:48:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.458+0000 7fa293600640 1 -- 172.21.15.16:0/2047621631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa28c064db0 con 0x7fa28c155a20 2024-08-17T02:48:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.459+0000 7fa288c00640 1 -- 172.21.15.16:0/2047621631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+12391 (secure 0 0 0) 0x7fa274017020 con 0x7fa28c155a20 2024-08-17T02:48:51.460 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:51.461 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":46,"fsid":"4a1fca9f-2f4a-44ff-8c68-dcf8d1ec9778","created":"2024-08-17T02:45:01.694817+0000","modified":"2024-08-17T02:48:46.649763+0000","last_up_change":"2024-08-17T02:45:12.982747+0000","last_in_change":"2024-08-17T02:45:04.578707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-17T02:45:14.291903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"6812bdf0-dd96-47a4-aa32-d93f99459266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":37182},{"type":"v1","addr":"172.21.15.16:6825","nonce":37182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":37182},{"type":"v1","addr":"172.21.15.16:6827","nonce":37182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":37182},{"type":"v1","addr":"172.21.15.16:6831","nonce":37182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":37182},{"type":"v1","addr":"172.21.15.16:6829","nonce":37182}]},"public_addr":"172.21.15.16:6825/37182","cluster_addr":"172.21.15.16:6827/37182","heartbeat_back_addr":"172.21.15.16:6831/37182","heartbeat_front_addr":"172.21.15.16:6829/37182","state":["exists","up"]},{"osd":1,"uuid":"591698cf-7733-47dd-9084-c5876b53bbc5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":37179},{"type":"v1","addr":"172.21.15.16:6801","nonce":37179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":37179},{"type":"v1","addr":"172.21.15.16:6803","nonce":37179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":37179},{"type":"v1","addr":"172.21.15.16:6807","nonce":37179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":37179},{"type":"v1","addr":"172.21.15.16:6805","nonce":37179}]},"public_addr":"172.21.15.16:6801/37179","cluster_addr":"172.21.15.16:6803/37179","heartbeat_back_addr":"172.21.15.16:6807/37179","heartbeat_front_addr":"172.21.15.16:6805/37179","state":["exists","up"]},{"osd":2,"uuid":"b8f63e22-f125-40d2-b4bd-065a24d2a7e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":37183},{"type":"v1","addr":"172.21.15.16:6817","nonce":37183}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":37183},{"type":"v1","addr":"172.21.15.16:6819","nonce":37183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":37183},{"type":"v1","addr":"172.21.15.16:6823","nonce":37183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":37183},{"type":"v1","addr":"172.21.15.16:6821","nonce":37183}]},"public_addr":"172.21.15.16:6817/37183","cluster_addr":"172.21.15.16:6819/37183","heartbeat_back_addr":"172.21.15.16:6823/37183","heartbeat_front_addr":"172.21.15.16:6821/37183","state":["exists","up"]},{"osd":3,"uuid":"5f06efd4-549c-4cd6-a464-b380c84840f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":37181},{"type":"v1","addr":"172.21.15.16:6809","nonce":37181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":37181},{"type":"v1","addr":"172.21.15.16:6811","nonce":37181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":37181},{"type":"v1","addr":"172.21.15.16:6815","nonce":37181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":37181},{"type":"v1","addr":"172.21.15.16:6813","nonce":37181}]},"public_addr":"172.21.15.16:6809/37181","cluster_addr":"172.21.15.16:6811/37181","heartbeat_back_addr":"172.21.15.16:6815/37181","heartbeat_front_addr":"172.21.15.16:6813/37181","state":["exists","up"]},{"osd":4,"uuid":"5a4cc2a0-711a-4101-bea6-2a313b9bf80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":37611},{"type":"v1","addr":"172.21.15.142:6820","nonce":37611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6821","nonce":37611},{"type":"v1","addr":"172.21.15.142:6823","nonce":37611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6829","nonce":37611},{"type":"v1","addr":"172.21.15.142:6831","nonce":37611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6825","nonce":37611},{"type":"v1","addr":"172.21.15.142:6827","nonce":37611}]},"public_addr":"172.21.15.142:6820/37611","cluster_addr":"172.21.15.142:6823/37611","heartbeat_back_addr":"172.21.15.142:6831/37611","heartbeat_front_addr":"172.21.15.142:6827/37611","state":["exists","up"]},{"osd":5,"uuid":"6235a39a-b1de-4308-8468-0cf430a64f4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":37609},{"type":"v1","addr":"172.21.15.142:6801","nonce":37609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":37609},{"type":"v1","addr":"172.21.15.142:6803","nonce":37609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":37609},{"type":"v1","addr":"172.21.15.142:6807","nonce":37609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":37609},{"type":"v1","addr":"172.21.15.142:6805","nonce":37609}]},"public_addr":"172.21.15.142:6801/37609","cluster_addr":"172.21.15.142:6803/37609","heartbeat_back_addr":"172.21.15.142:6807/37609","heartbeat_front_addr":"172.21.15.142:6805/37609","state":["exists","up"]},{"osd":6,"uuid":"178547a0-450d-42c8-80b0-6e887a5877aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":37610},{"type":"v1","addr":"172.21.15.142:6809","nonce":37610}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":37610},{"type":"v1","addr":"172.21.15.142:6811","nonce":37610}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":37610},{"type":"v1","addr":"172.21.15.142:6815","nonce":37610}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":37610},{"type":"v1","addr":"172.21.15.142:6813","nonce":37610}]},"public_addr":"172.21.15.142:6809/37610","cluster_addr":"172.21.15.142:6811/37610","heartbeat_back_addr":"172.21.15.142:6815/37610","heartbeat_front_addr":"172.21.15.142:6813/37610","state":["exists","up"]},{"osd":7,"uuid":"7e5915cc-7027-4063-ac18-385c4354a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":37634},{"type":"v1","addr":"172.21.15.142:6817","nonce":37634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6819","nonce":37634},{"type":"v1","addr":"172.21.15.142:6822","nonce":37634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":37634},{"type":"v1","addr":"172.21.15.142:6830","nonce":37634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":37634},{"type":"v1","addr":"172.21.15.142:6826","nonce":37634}]},"public_addr":"172.21.15.142:6817/37634","cluster_addr":"172.21.15.142:6822/37634","heartbeat_back_addr":"172.21.15.142:6830/37634","heartbeat_front_addr":"172.21.15.142:6826/37634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:10.010990+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.863408+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.985827+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.925102+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.912838+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.913755+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.828284+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-08-17T02:45:09.857157+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6834/3115083427":"2024-08-18T02:48:43.481423+0000","172.21.15.16:6832/3115083427":"2024-08-18T02:48:43.481423+0000","172.21.15.16:6835/219684854":"2024-08-18T02:48:36.607854+0000","172.21.15.16:6833/219684854":"2024-08-18T02:48:36.607854+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-17T02:48:51.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.461+0000 7fa293600640 1 -- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa268065150 msgr2=0x7fa268067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.461+0000 7fa293600640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa268065150 0x7fa268067600 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009d20 tx=0x7fa27c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.461+0000 7fa293600640 1 -- 172.21.15.16:0/2047621631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c155a20 msgr2=0x7fa28c146ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.461+0000 7fa293600640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c155a20 0x7fa28c146ff0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa27400c4c0 tx=0x7fa274007ef0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.461+0000 7fa293600640 1 -- 172.21.15.16:0/2047621631 shutdown_connections 2024-08-17T02:48:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.461+0000 7fa293600640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa268065150 0x7fa268067600 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.461+0000 7fa293600640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa28c155df0 0x7fa28c147530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.461+0000 7fa293600640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c155a20 0x7fa28c146ff0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.461+0000 7fa293600640 1 --2- 172.21.15.16:0/2047621631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa28c06df00 0x7fa28c152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.461+0000 7fa293600640 1 -- 172.21.15.16:0/2047621631 >> 172.21.15.16:0/2047621631 conn(0x7fa28c0cb9b0 msgr2=0x7fa28c06d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.462+0000 7fa293600640 1 -- 172.21.15.16:0/2047621631 shutdown_connections 2024-08-17T02:48:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:51.462+0000 7fa293600640 1 -- 172.21.15.16:0/2047621631 wait complete. 2024-08-17T02:48:52.478 INFO:tasks.ceph:Scrubbing osd.0 2024-08-17T02:48:52.478 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:48:52.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.644+0000 7fad84c00640 1 Processor -- start 2024-08-17T02:48:52.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.644+0000 7fad84c00640 1 -- start start 2024-08-17T02:48:52.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.646+0000 7fad84c00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad8006e5f0 0x7fad800c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.646+0000 7fad84c00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fad80156c60 0x7fad800c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.646+0000 7fad84c00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad800ae9b0 con 0x7fad80156890 2024-08-17T02:48:52.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.646+0000 7fad84c00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad800c8bb0 con 0x7fad8006e5f0 2024-08-17T02:48:52.647 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.646+0000 7fad84c00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad800aa730 con 0x7fad80156c60 2024-08-17T02:48:52.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.646+0000 7fad77e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad8006e5f0 0x7fad800c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.646+0000 7fad77e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad8006e5f0 0x7fad800c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39702/0 (socket says 172.21.15.16:39702) 2024-08-17T02:48:52.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.646+0000 7fad77e00640 1 -- 172.21.15.16:0/96742233 learned_addr learned my addr 172.21.15.16:0/96742233 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:52.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.646+0000 7fad7e800640 1 --2- 172.21.15.16:0/96742233 >> v2:172.21.15.142:3300/0 conn(0x7fad80156c60 0x7fad800c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.647+0000 7fad77e00640 1 -- 172.21.15.16:0/96742233 >> v2:172.21.15.142:3300/0 conn(0x7fad80156c60 msgr2=0x7fad800c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.647+0000 7fad77e00640 1 --2- 172.21.15.16:0/96742233 >> v2:172.21.15.142:3300/0 conn(0x7fad80156c60 0x7fad800c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.647+0000 7fad77e00640 1 -- 172.21.15.16:0/96742233 >> v1:172.21.15.142:6789/0 conn(0x7fad80156890 legacy=0x7fad8006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.647+0000 7fad77e00640 1 -- 172.21.15.16:0/96742233 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad800ab320 con 0x7fad8006e5f0 2024-08-17T02:48:52.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.648+0000 7fad77e00640 1 --2- 172.21.15.16:0/96742233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8006e5f0 0x7fad800c8130 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fad8004bdc0 tx=0x7fad680091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53270cdd04e0ae2 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.648+0000 7fad7d400640 1 -- 172.21.15.16:0/96742233 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6802add0 con 0x7fad8006e5f0 2024-08-17T02:48:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.648+0000 7fad7d400640 1 -- 172.21.15.16:0/96742233 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad68032440 con 0x7fad8006e5f0 2024-08-17T02:48:52.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.648+0000 7fad7d400640 1 -- 172.21.15.16:0/96742233 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68032820 con 0x7fad8006e5f0 2024-08-17T02:48:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.648+0000 7fad84c00640 1 -- 172.21.15.16:0/96742233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8006e5f0 msgr2=0x7fad800c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.648+0000 7fad84c00640 1 --2- 172.21.15.16:0/96742233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8006e5f0 0x7fad800c8130 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fad8004bdc0 tx=0x7fad680091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.648+0000 7fad84c00640 1 -- 172.21.15.16:0/96742233 shutdown_connections 2024-08-17T02:48:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.649+0000 7fad84c00640 1 --2- 172.21.15.16:0/96742233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad8006e5f0 0x7fad800c8130 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.649+0000 7fad84c00640 1 --2- 172.21.15.16:0/96742233 >> v2:172.21.15.142:3300/0 conn(0x7fad80156c60 0x7fad800c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.649+0000 7fad84c00640 1 -- 172.21.15.16:0/96742233 >> 172.21.15.16:0/96742233 conn(0x7fad80067750 msgr2=0x7fad8006c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.649+0000 7fad84c00640 1 -- 172.21.15.16:0/96742233 shutdown_connections 2024-08-17T02:48:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.649+0000 7fad84c00640 1 -- 172.21.15.16:0/96742233 wait complete. 2024-08-17T02:48:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.650+0000 7fad84c00640 1 Processor -- start 2024-08-17T02:48:52.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.650+0000 7fad84c00640 1 -- start start 2024-08-17T02:48:52.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.651+0000 7fad84c00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad8006e5f0 0x7fad8014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.651+0000 7fad84c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80156890 0x7fad8014d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.651+0000 7fad84c00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad80156c60 0x7fad80168ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.651+0000 7fad84c00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad80169eb0 con 0x7fad80156c60 2024-08-17T02:48:52.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.651+0000 7fad84c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad801698f0 con 0x7fad80156890 2024-08-17T02:48:52.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.651+0000 7fad84c00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad80169bd0 con 0x7fad8006e5f0 2024-08-17T02:48:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.651+0000 7fad7e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad80156c60 0x7fad80168ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.651+0000 7fad7e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad80156c60 0x7fad80168ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39708/0 (socket says 172.21.15.16:39708) 2024-08-17T02:48:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.651+0000 7fad7e800640 1 -- 172.21.15.16:0/3440698397 learned_addr learned my addr 172.21.15.16:0/3440698397 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.651+0000 7fad7de00640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad8006e5f0 0x7fad8014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.651+0000 7fad77e00640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80156890 0x7fad8014d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.652+0000 7fad7e800640 1 -- 172.21.15.16:0/3440698397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad8006e5f0 msgr2=0x7fad8014fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.652+0000 7fad7e800640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fad8006e5f0 0x7fad8014fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.652+0000 7fad7e800640 1 -- 172.21.15.16:0/3440698397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80156890 msgr2=0x7fad8014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.652+0000 7fad7e800640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad80156890 0x7fad8014d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.652+0000 7fad7e800640 1 -- 172.21.15.16:0/3440698397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad68009450 con 0x7fad80156c60 2024-08-17T02:48:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.652+0000 7fad7e800640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad80156c60 0x7fad80168ea0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fad78009df0 tx=0x7fad7800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.652+0000 7fad75600640 1 -- 172.21.15.16:0/3440698397 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7800cd40 con 0x7fad80156c60 2024-08-17T02:48:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.652+0000 7fad75600640 1 -- 172.21.15.16:0/3440698397 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7800f040 con 0x7fad80156c60 2024-08-17T02:48:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.652+0000 7fad75600640 1 -- 172.21.15.16:0/3440698397 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad780146d0 con 0x7fad80156c60 2024-08-17T02:48:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.652+0000 7fad84c00640 1 -- 172.21.15.16:0/3440698397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8014f660 con 0x7fad80156c60 2024-08-17T02:48:52.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.653+0000 7fad84c00640 1 -- 172.21.15.16:0/3440698397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad801503a0 con 0x7fad80156c60 2024-08-17T02:48:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.653+0000 7fad84c00640 1 -- 172.21.15.16:0/3440698397 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad8014f4c0 con 0x7fad80156c60 2024-08-17T02:48:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.654+0000 7fad75600640 1 -- 172.21.15.16:0/3440698397 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fad78014a80 con 0x7fad80156c60 2024-08-17T02:48:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.655+0000 7fad75600640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fad5c065150 0x7fad5c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.655+0000 7fad75600640 1 -- 172.21.15.16:0/3440698397 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fad78082b50 con 0x7fad80156c60 2024-08-17T02:48:52.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.655+0000 7fad75600640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] conn(0x7fad5c06b040 0x7fad5c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.655+0000 7fad75600640 1 -- 172.21.15.16:0/3440698397 --> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad5c06db00 con 0x7fad5c06b040 2024-08-17T02:48:52.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.655+0000 7fad75600640 1 -- 172.21.15.16:0/3440698397 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad78087220 con 0x7fad80156c60 2024-08-17T02:48:52.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.656+0000 7fad7de00640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fad5c065150 0x7fad5c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.656+0000 7fad77e00640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] conn(0x7fad5c06b040 0x7fad5c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.656+0000 7fad77e00640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] conn(0x7fad5c06b040 0x7fad5c06d450 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:52.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.656+0000 7fad7de00640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fad5c065150 0x7fad5c067600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fad80150680 tx=0x7fad70009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:52.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.657+0000 7fad75600640 1 -- 172.21.15.16:0/3440698397 <== osd.0 v2:172.21.15.16:6824/37182 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fad5c06db00 con 0x7fad5c06b040 2024-08-17T02:48:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.670+0000 7fad84c00640 1 -- 172.21.15.16:0/3440698397 --> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fad800710c0 con 0x7fad5c06b040 2024-08-17T02:48:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.670+0000 7fad75600640 1 -- 172.21.15.16:0/3440698397 <== osd.0 v2:172.21.15.16:6824/37182 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fad800710c0 con 0x7fad5c06b040 2024-08-17T02:48:52.671 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-17T02:48:52.672 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:48:52.672 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-17T02:48:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.671+0000 7fad84c00640 1 -- 172.21.15.16:0/3440698397 >> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] conn(0x7fad5c06b040 msgr2=0x7fad5c06d450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.671+0000 7fad84c00640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.16:6824/37182,v1:172.21.15.16:6825/37182] conn(0x7fad5c06b040 0x7fad5c06d450 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.671+0000 7fad84c00640 1 -- 172.21.15.16:0/3440698397 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fad5c065150 msgr2=0x7fad5c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.671+0000 7fad84c00640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fad5c065150 0x7fad5c067600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fad80150680 tx=0x7fad70009040 comp rx=0 tx=0).stop 2024-08-17T02:48:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.671+0000 7fad84c00640 1 -- 172.21.15.16:0/3440698397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad80156c60 msgr2=0x7fad80168ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.671+0000 7fad84c00640 1 --2- 172.21.15.16:0/3440698397 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad80156c60 0x7fad80168ea0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fad78009df0 tx=0x7fad7800b040 comp rx=0 tx=0).stop 2024-08-17T02:48:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.671+0000 7fad7e800640 1 -- 172.21.15.16:0/3440698397 reap_dead start 2024-08-17T02:48:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.671+0000 7fad84c00640 1 -- 172.21.15.16:0/3440698397 shutdown_connections 2024-08-17T02:48:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.671+0000 7fad84c00640 1 -- 172.21.15.16:0/3440698397 >> 172.21.15.16:0/3440698397 conn(0x7fad80067750 msgr2=0x7fad80154da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.671+0000 7fad84c00640 1 -- 172.21.15.16:0/3440698397 shutdown_connections 2024-08-17T02:48:52.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.671+0000 7fad84c00640 1 -- 172.21.15.16:0/3440698397 wait complete. 2024-08-17T02:48:52.688 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-08-17T02:48:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.838+0000 7f4837200640 1 Processor -- start 2024-08-17T02:48:52.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.838+0000 7f4837200640 1 -- start start 2024-08-17T02:48:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.840+0000 7f4837200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f48380cc010 0x7f48380ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.840+0000 7f4837200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48380a98e0 0x7f48380afe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.840+0000 7f4837200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48380ab950 con 0x7f48380b03d0 2024-08-17T02:48:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.840+0000 7f4837200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48380abc30 con 0x7f48380cc010 2024-08-17T02:48:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.840+0000 7f4837200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48380abf10 con 0x7f48380a98e0 2024-08-17T02:48:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.840+0000 7f4835400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48380a98e0 0x7f48380afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.840+0000 7f4835e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f48380cc010 0x7f48380ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.840+0000 7f4835400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48380a98e0 0x7f48380afe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3301/0 says I am v2:172.21.15.16:45738/0 (socket says 172.21.15.16:45738) 2024-08-17T02:48:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.840+0000 7f4835400640 1 -- 172.21.15.16:0/4159836792 learned_addr learned my addr 172.21.15.16:0/4159836792 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.841+0000 7f4835e00640 1 -- 172.21.15.16:0/4159836792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48380a98e0 msgr2=0x7f48380afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.841+0000 7f4835e00640 1 --2- 172.21.15.16:0/4159836792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48380a98e0 0x7f48380afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.841+0000 7f4835e00640 1 -- 172.21.15.16:0/4159836792 >> v1:172.21.15.142:6789/0 conn(0x7f48380b03d0 legacy=0x7f483815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.841+0000 7f4835e00640 1 -- 172.21.15.16:0/4159836792 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48380af970 con 0x7f48380cc010 2024-08-17T02:48:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.841+0000 7f4835400640 1 --2- 172.21.15.16:0/4159836792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48380a98e0 0x7f48380afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-17T02:48:52.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.842+0000 7f4835e00640 1 --2- 172.21.15.16:0/4159836792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48380cc010 0x7f48380ab2b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4828009850 tx=0x7f48280090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c9a6dedeb01773 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.842+0000 7f4834a00640 1 -- 172.21.15.16:0/4159836792 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482802ad00 con 0x7f48380cc010 2024-08-17T02:48:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.842+0000 7f4834a00640 1 -- 172.21.15.16:0/4159836792 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482802ae60 con 0x7f48380cc010 2024-08-17T02:48:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.842+0000 7f4834a00640 1 -- 172.21.15.16:0/4159836792 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4828032670 con 0x7f48380cc010 2024-08-17T02:48:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.843+0000 7f4837200640 1 -- 172.21.15.16:0/4159836792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48380cc010 msgr2=0x7f48380ab2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.843+0000 7f4837200640 1 --2- 172.21.15.16:0/4159836792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48380cc010 0x7f48380ab2b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4828009850 tx=0x7f48280090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.843+0000 7f4837200640 1 -- 172.21.15.16:0/4159836792 shutdown_connections 2024-08-17T02:48:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.843+0000 7f4837200640 1 --2- 172.21.15.16:0/4159836792 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48380a98e0 0x7f48380afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.843+0000 7f4837200640 1 --2- 172.21.15.16:0/4159836792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48380cc010 0x7f48380ab2b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.843+0000 7f4837200640 1 -- 172.21.15.16:0/4159836792 >> 172.21.15.16:0/4159836792 conn(0x7f4838067de0 msgr2=0x7f48380681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.843+0000 7f4837200640 1 -- 172.21.15.16:0/4159836792 shutdown_connections 2024-08-17T02:48:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.843+0000 7f4837200640 1 -- 172.21.15.16:0/4159836792 wait complete. 2024-08-17T02:48:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.844+0000 7f4837200640 1 Processor -- start 2024-08-17T02:48:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.844+0000 7f4837200640 1 -- start start 2024-08-17T02:48:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.845+0000 7f4837200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48380a98e0 0x7f4838155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.845+0000 7f4837200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48380b03d0 0x7f483815a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.845+0000 7f4837200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48380cc010 0x7f48380c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.845+0000 7f4837200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483815b730 con 0x7f48380cc010 2024-08-17T02:48:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.845+0000 7f4837200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483815ab20 con 0x7f48380b03d0 2024-08-17T02:48:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.845+0000 7f4837200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483815ae00 con 0x7f48380a98e0 2024-08-17T02:48:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.846+0000 7f4836800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48380cc010 0x7f48380c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.846+0000 7f4836800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48380cc010 0x7f48380c41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39716/0 (socket says 172.21.15.16:39716) 2024-08-17T02:48:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.846+0000 7f4836800640 1 -- 172.21.15.16:0/2595612305 learned_addr learned my addr 172.21.15.16:0/2595612305 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.846+0000 7f4835e00640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48380a98e0 0x7f4838155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.846+0000 7f4835400640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48380b03d0 0x7f483815a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.846+0000 7f4836800640 1 -- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48380a98e0 msgr2=0x7f4838155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.846+0000 7f4836800640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48380a98e0 0x7f4838155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.846+0000 7f4836800640 1 -- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48380b03d0 msgr2=0x7f483815a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.846+0000 7f4836800640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48380b03d0 0x7f483815a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.846+0000 7f4836800640 1 -- 172.21.15.16:0/2595612305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4824009450 con 0x7f48380cc010 2024-08-17T02:48:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.846+0000 7f4836800640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48380cc010 0x7f48380c41d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f483000c860 tx=0x7f4830001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.847+0000 7f482d600640 1 -- 172.21.15.16:0/2595612305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483000ce50 con 0x7f48380cc010 2024-08-17T02:48:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.847+0000 7f482d600640 1 -- 172.21.15.16:0/2595612305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483000a850 con 0x7f48380cc010 2024-08-17T02:48:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.847+0000 7f482d600640 1 -- 172.21.15.16:0/2595612305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483000ac10 con 0x7f48380cc010 2024-08-17T02:48:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.847+0000 7f4837200640 1 -- 172.21.15.16:0/2595612305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4828009390 con 0x7f48380cc010 2024-08-17T02:48:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.847+0000 7f4837200640 1 -- 172.21.15.16:0/2595612305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483815b3c0 con 0x7f48380cc010 2024-08-17T02:48:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.848+0000 7f4837200640 1 -- 172.21.15.16:0/2595612305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483815b8e0 con 0x7f48380cc010 2024-08-17T02:48:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.849+0000 7f482d600640 1 -- 172.21.15.16:0/2595612305 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f4830012630 con 0x7f48380cc010 2024-08-17T02:48:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.850+0000 7f482d600640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4818065150 0x7f4818067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.851+0000 7f482d600640 1 -- 172.21.15.16:0/2595612305 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f4830082350 con 0x7f48380cc010 2024-08-17T02:48:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.851+0000 7f4835e00640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4818065150 0x7f4818067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.852+0000 7f4835e00640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4818065150 0x7f4818067600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4828034000 tx=0x7f4828032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:52.855+0000 7f482d600640 1 -- 172.21.15.16:0/2595612305 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f483815b8e0 con 0x7f48380cc010 2024-08-17T02:48:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.038+0000 7f4837200640 1 -- 172.21.15.16:0/2595612305 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f4838071050 con 0x7f4818065150 2024-08-17T02:48:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.038+0000 7f482d600640 1 -- 172.21.15.16:0/2595612305 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4838071050 con 0x7f4818065150 2024-08-17T02:48:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 0 to deep-scrub 2024-08-17T02:48:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.041+0000 7f4837200640 1 -- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4818065150 msgr2=0x7f4818067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.041+0000 7f4837200640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4818065150 0x7f4818067600 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4828034000 tx=0x7f4828032000 comp rx=0 tx=0).stop 2024-08-17T02:48:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.041+0000 7f4837200640 1 -- 172.21.15.16:0/2595612305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48380cc010 msgr2=0x7f48380c41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.041+0000 7f4837200640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48380cc010 0x7f48380c41d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f483000c860 tx=0x7f4830001690 comp rx=0 tx=0).stop 2024-08-17T02:48:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.041+0000 7f4837200640 1 -- 172.21.15.16:0/2595612305 shutdown_connections 2024-08-17T02:48:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.041+0000 7f4837200640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f4818065150 0x7f4818067600 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.041+0000 7f4837200640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48380cc010 0x7f48380c41d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.041+0000 7f4837200640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48380b03d0 0x7f483815a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.041+0000 7f4837200640 1 --2- 172.21.15.16:0/2595612305 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f48380a98e0 0x7f4838155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.041+0000 7f4837200640 1 -- 172.21.15.16:0/2595612305 >> 172.21.15.16:0/2595612305 conn(0x7f4838067de0 msgr2=0x7f4838158440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.041+0000 7f4837200640 1 -- 172.21.15.16:0/2595612305 shutdown_connections 2024-08-17T02:48:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.041+0000 7f4837200640 1 -- 172.21.15.16:0/2595612305 wait complete. 2024-08-17T02:48:53.056 INFO:tasks.ceph:Scrubbing osd.1 2024-08-17T02:48:53.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:48:53.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.210+0000 7f5331200640 1 Processor -- start 2024-08-17T02:48:53.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.210+0000 7f5331200640 1 -- start start 2024-08-17T02:48:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.212+0000 7f5331200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f532c06e5f0 0x7f532c0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.212+0000 7f5331200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c0ae9b0 con 0x7f532c156890 2024-08-17T02:48:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.212+0000 7f5331200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c0ab320 con 0x7f532c156c60 2024-08-17T02:48:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.212+0000 7f5331200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c0aa730 con 0x7f532c06e5f0 2024-08-17T02:48:53.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.212+0000 7f5329400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f532c06e5f0 0x7f532c0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.212+0000 7f5329400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f532c06e5f0 0x7f532c0c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39724/0 (socket says 172.21.15.16:39724) 2024-08-17T02:48:53.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.212+0000 7f5329400640 1 -- 172.21.15.16:0/3054763712 learned_addr learned my addr 172.21.15.16:0/3054763712 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:53.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.212+0000 7f5329e00640 1 --1- 172.21.15.16:0/3054763712 >> v1:172.21.15.16:6789/0 conn(0x7f532c156890 0x7f532c06dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35016/0 (socket says 172.21.15.16:35016) 2024-08-17T02:48:53.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.212+0000 7f5329400640 1 -- 172.21.15.16:0/3054763712 >> v1:172.21.15.142:6789/0 conn(0x7f532c156c60 legacy=0x7f532c0ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.212+0000 7f5329400640 1 -- 172.21.15.16:0/3054763712 >> v1:172.21.15.16:6789/0 conn(0x7f532c156890 legacy=0x7f532c06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.212+0000 7f5328a00640 1 -- 172.21.15.16:0/3054763712 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157036680 0 0) 0x7f532c0ae9b0 con 0x7f532c156890 2024-08-17T02:48:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.212+0000 7f5329400640 1 -- 172.21.15.16:0/3054763712 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c0aa9f0 con 0x7f532c06e5f0 2024-08-17T02:48:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.213+0000 7f5329400640 1 --2- 172.21.15.16:0/3054763712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c06e5f0 0x7f532c0c8130 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5318009910 tx=0x7f53180091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38259b1b7971ad7 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.214+0000 7f5328a00640 1 -- 172.21.15.16:0/3054763712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531802ae60 con 0x7f532c06e5f0 2024-08-17T02:48:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.214+0000 7f5328a00640 1 -- 172.21.15.16:0/3054763712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5318032440 con 0x7f532c06e5f0 2024-08-17T02:48:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.214+0000 7f5328a00640 1 -- 172.21.15.16:0/3054763712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180327e0 con 0x7f532c06e5f0 2024-08-17T02:48:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.214+0000 7f5331200640 1 -- 172.21.15.16:0/3054763712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c06e5f0 msgr2=0x7f532c0c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.214+0000 7f5331200640 1 --2- 172.21.15.16:0/3054763712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c06e5f0 0x7f532c0c8130 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5318009910 tx=0x7f53180091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.214+0000 7f5331200640 1 -- 172.21.15.16:0/3054763712 shutdown_connections 2024-08-17T02:48:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.214+0000 7f5331200640 1 --2- 172.21.15.16:0/3054763712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c06e5f0 0x7f532c0c8130 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.214+0000 7f5331200640 1 -- 172.21.15.16:0/3054763712 >> 172.21.15.16:0/3054763712 conn(0x7f532c067750 msgr2=0x7f532c06c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.214+0000 7f5331200640 1 -- 172.21.15.16:0/3054763712 shutdown_connections 2024-08-17T02:48:53.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.214+0000 7f5331200640 1 -- 172.21.15.16:0/3054763712 wait complete. 2024-08-17T02:48:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.216+0000 7f5331200640 1 Processor -- start 2024-08-17T02:48:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.216+0000 7f5331200640 1 -- start start 2024-08-17T02:48:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.217+0000 7f5331200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f532c06e5f0 0x7f532c16a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.217+0000 7f5331200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c156890 0x7f532c167070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.217+0000 7f5331200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c156c60 0x7f532c14e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.217+0000 7f5331200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c14f490 con 0x7f532c156c60 2024-08-17T02:48:53.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.217+0000 7f5331200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c148e00 con 0x7f532c156890 2024-08-17T02:48:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.217+0000 7f5331200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c14ebf0 con 0x7f532c06e5f0 2024-08-17T02:48:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.217+0000 7f532a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c156c60 0x7f532c14e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.217+0000 7f532a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c156c60 0x7f532c14e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39732/0 (socket says 172.21.15.16:39732) 2024-08-17T02:48:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.217+0000 7f532a800640 1 -- 172.21.15.16:0/173134648 learned_addr learned my addr 172.21.15.16:0/173134648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.217+0000 7f5329400640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c156890 0x7f532c167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.217+0000 7f5329e00640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f532c06e5f0 0x7f532c16a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.218+0000 7f532a800640 1 -- 172.21.15.16:0/173134648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f532c06e5f0 msgr2=0x7f532c16a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.218+0000 7f532a800640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f532c06e5f0 0x7f532c16a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.218+0000 7f532a800640 1 -- 172.21.15.16:0/173134648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c156890 msgr2=0x7f532c167070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.218+0000 7f532a800640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f532c156890 0x7f532c167070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.218+0000 7f532a800640 1 -- 172.21.15.16:0/173134648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5318009450 con 0x7f532c156c60 2024-08-17T02:48:53.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.218+0000 7f532a800640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c156c60 0x7f532c14e380 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f532400c7f0 tx=0x7f53240015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.218+0000 7f5321600640 1 -- 172.21.15.16:0/173134648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532400a640 con 0x7f532c156c60 2024-08-17T02:48:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.218+0000 7f5321600640 1 -- 172.21.15.16:0/173134648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532400a7c0 con 0x7f532c156c60 2024-08-17T02:48:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.218+0000 7f5321600640 1 -- 172.21.15.16:0/173134648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532400ab80 con 0x7f532c156c60 2024-08-17T02:48:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.218+0000 7f5331200640 1 -- 172.21.15.16:0/173134648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c14eed0 con 0x7f532c156c60 2024-08-17T02:48:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.218+0000 7f5331200640 1 -- 172.21.15.16:0/173134648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c14e8c0 con 0x7f532c156c60 2024-08-17T02:48:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.219+0000 7f5331200640 1 -- 172.21.15.16:0/173134648 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f532c0710c0 con 0x7f532c156c60 2024-08-17T02:48:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.220+0000 7f5321600640 1 -- 172.21.15.16:0/173134648 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f5324003030 con 0x7f532c156c60 2024-08-17T02:48:53.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.221+0000 7f5321600640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f530c065150 0x7f530c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.221+0000 7f5321600640 1 -- 172.21.15.16:0/173134648 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f5324082260 con 0x7f532c156c60 2024-08-17T02:48:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.221+0000 7f5321600640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] conn(0x7f530c06b040 0x7f530c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.221+0000 7f5321600640 1 -- 172.21.15.16:0/173134648 --> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f530c06db00 con 0x7f530c06b040 2024-08-17T02:48:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.221+0000 7f5321600640 1 -- 172.21.15.16:0/173134648 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53240847c0 con 0x7f532c156c60 2024-08-17T02:48:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.221+0000 7f5329e00640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f530c065150 0x7f530c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.222+0000 7f5329400640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] conn(0x7f530c06b040 0x7f530c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.222+0000 7f5329e00640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f530c065150 0x7f530c067600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f532c06abd0 tx=0x7f531c007ca0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.222+0000 7f5329400640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] conn(0x7f530c06b040 0x7f530c06d450 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.223+0000 7f5321600640 1 -- 172.21.15.16:0/173134648 <== osd.1 v2:172.21.15.16:6800/37179 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f530c06db00 con 0x7f530c06b040 2024-08-17T02:48:53.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.236+0000 7f5331200640 1 -- 172.21.15.16:0/173134648 --> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f532c070740 con 0x7f530c06b040 2024-08-17T02:48:53.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.236+0000 7f5321600640 1 -- 172.21.15.16:0/173134648 <== osd.1 v2:172.21.15.16:6800/37179 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f532c070740 con 0x7f530c06b040 2024-08-17T02:48:53.238 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-17T02:48:53.238 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:48:53.238 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-17T02:48:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.236+0000 7f5331200640 1 -- 172.21.15.16:0/173134648 >> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] conn(0x7f530c06b040 msgr2=0x7f530c06d450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.236+0000 7f5331200640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.16:6800/37179,v1:172.21.15.16:6801/37179] conn(0x7f530c06b040 0x7f530c06d450 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.237+0000 7f5331200640 1 -- 172.21.15.16:0/173134648 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f530c065150 msgr2=0x7f530c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.237+0000 7f5331200640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f530c065150 0x7f530c067600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f532c06abd0 tx=0x7f531c007ca0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.237+0000 7f5331200640 1 -- 172.21.15.16:0/173134648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c156c60 msgr2=0x7f532c14e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.237+0000 7f5331200640 1 --2- 172.21.15.16:0/173134648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f532c156c60 0x7f532c14e380 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f532400c7f0 tx=0x7f53240015f0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.237+0000 7f532a800640 1 -- 172.21.15.16:0/173134648 reap_dead start 2024-08-17T02:48:53.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.237+0000 7f5331200640 1 -- 172.21.15.16:0/173134648 shutdown_connections 2024-08-17T02:48:53.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.237+0000 7f5331200640 1 -- 172.21.15.16:0/173134648 >> 172.21.15.16:0/173134648 conn(0x7f532c067750 msgr2=0x7f532c154a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:53.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.237+0000 7f5331200640 1 -- 172.21.15.16:0/173134648 shutdown_connections 2024-08-17T02:48:53.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.237+0000 7f5331200640 1 -- 172.21.15.16:0/173134648 wait complete. 2024-08-17T02:48:53.252 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-08-17T02:48:53.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.403+0000 7f14d5e00640 1 Processor -- start 2024-08-17T02:48:53.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.403+0000 7f14d5e00640 1 -- start start 2024-08-17T02:48:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.404+0000 7f14d5e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14d0155df0 0x7f14d00adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.404+0000 7f14d5e00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d0156670 con 0x7f14d006df00 2024-08-17T02:48:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.404+0000 7f14d5e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d006e5e0 con 0x7f14d0155a20 2024-08-17T02:48:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.404+0000 7f14d5e00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d00b0350 con 0x7f14d0155df0 2024-08-17T02:48:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.405+0000 7f14cf400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14d0155df0 0x7f14d00adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.405+0000 7f14cf400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14d0155df0 0x7f14d00adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39738/0 (socket says 172.21.15.16:39738) 2024-08-17T02:48:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.405+0000 7f14cf400640 1 -- 172.21.15.16:0/3989344731 learned_addr learned my addr 172.21.15.16:0/3989344731 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.405+0000 7f14ce000640 1 --1- 172.21.15.16:0/3989344731 >> v1:172.21.15.16:6789/0 conn(0x7f14d006df00 0x7f14d0156b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35030/0 (socket says 172.21.15.16:35030) 2024-08-17T02:48:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.405+0000 7f14cf400640 1 -- 172.21.15.16:0/3989344731 >> v1:172.21.15.142:6789/0 conn(0x7f14d0155a20 legacy=0x7f14d006d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.405+0000 7f14cf400640 1 -- 172.21.15.16:0/3989344731 >> v1:172.21.15.16:6789/0 conn(0x7f14d006df00 legacy=0x7f14d0156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.405+0000 7f14cf400640 1 -- 172.21.15.16:0/3989344731 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d006e8c0 con 0x7f14d0155df0 2024-08-17T02:48:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.405+0000 7f14cd600640 1 -- 172.21.15.16:0/3989344731 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52518843 0 0) 0x7f14d0156670 con 0x7f14d006df00 2024-08-17T02:48:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.406+0000 7f14cf400640 1 --2- 172.21.15.16:0/3989344731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d0155df0 0x7f14d00adf10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009f20 tx=0x7f14c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee98637af334bcd server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.406+0000 7f14cd600640 1 -- 172.21.15.16:0/3989344731 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8031400 con 0x7f14d0155df0 2024-08-17T02:48:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.406+0000 7f14cd600640 1 -- 172.21.15.16:0/3989344731 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c802d050 con 0x7f14d0155df0 2024-08-17T02:48:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.406+0000 7f14cd600640 1 -- 172.21.15.16:0/3989344731 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8031790 con 0x7f14d0155df0 2024-08-17T02:48:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.407+0000 7f14d5e00640 1 -- 172.21.15.16:0/3989344731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d0155df0 msgr2=0x7f14d00adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.407+0000 7f14d5e00640 1 --2- 172.21.15.16:0/3989344731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d0155df0 0x7f14d00adf10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009f20 tx=0x7f14c8009840 comp rx=0 tx=0).stop 2024-08-17T02:48:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.407+0000 7f14d5e00640 1 -- 172.21.15.16:0/3989344731 shutdown_connections 2024-08-17T02:48:53.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.407+0000 7f14d5e00640 1 --2- 172.21.15.16:0/3989344731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d0155df0 0x7f14d00adf10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.407+0000 7f14d5e00640 1 -- 172.21.15.16:0/3989344731 >> 172.21.15.16:0/3989344731 conn(0x7f14d00cb9b0 msgr2=0x7f14d00cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:53.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.407+0000 7f14d5e00640 1 -- 172.21.15.16:0/3989344731 shutdown_connections 2024-08-17T02:48:53.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.407+0000 7f14d5e00640 1 -- 172.21.15.16:0/3989344731 wait complete. 2024-08-17T02:48:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.408+0000 7f14d5e00640 1 Processor -- start 2024-08-17T02:48:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.408+0000 7f14d5e00640 1 -- start start 2024-08-17T02:48:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.409+0000 7f14d5e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d006df00 0x7f14d01464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.409+0000 7f14d5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0155a20 0x7f14d0151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.409+0000 7f14d5e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d0155df0 0x7f14d014f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.409+0000 7f14d5e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d01525d0 con 0x7f14d0155df0 2024-08-17T02:48:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.409+0000 7f14d5e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d01519c0 con 0x7f14d0155a20 2024-08-17T02:48:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.409+0000 7f14d5e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d0151ca0 con 0x7f14d006df00 2024-08-17T02:48:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.410+0000 7f14cf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d0155df0 0x7f14d014f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.410+0000 7f14cf400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d0155df0 0x7f14d014f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39754/0 (socket says 172.21.15.16:39754) 2024-08-17T02:48:53.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.410+0000 7f14cf400640 1 -- 172.21.15.16:0/1763860700 learned_addr learned my addr 172.21.15.16:0/1763860700 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:53.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.410+0000 7f14ce000640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0155a20 0x7f14d0151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.410+0000 7f14cea00640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d006df00 0x7f14d01464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.410+0000 7f14cf400640 1 -- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d006df00 msgr2=0x7f14d01464c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.410+0000 7f14cf400640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d006df00 0x7f14d01464c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.410+0000 7f14cf400640 1 -- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0155a20 msgr2=0x7f14d0151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.410+0000 7f14cf400640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0155a20 0x7f14d0151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.410+0000 7f14cf400640 1 -- 172.21.15.16:0/1763860700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c8009ae0 con 0x7f14d0155df0 2024-08-17T02:48:53.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.410+0000 7f14cea00640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d006df00 0x7f14d01464c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:53.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.410+0000 7f14cf400640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d0155df0 0x7f14d014f080 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009f20 tx=0x7f14c8036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.411+0000 7f14c6000640 1 -- 172.21.15.16:0/1763860700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8031e80 con 0x7f14d0155df0 2024-08-17T02:48:53.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.411+0000 7f14c6000640 1 -- 172.21.15.16:0/1763860700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c8008ad0 con 0x7f14d0155df0 2024-08-17T02:48:53.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.411+0000 7f14d5e00640 1 -- 172.21.15.16:0/1763860700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d0151f80 con 0x7f14d0155df0 2024-08-17T02:48:53.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.411+0000 7f14c6000640 1 -- 172.21.15.16:0/1763860700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8008e50 con 0x7f14d0155df0 2024-08-17T02:48:53.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.411+0000 7f14d5e00640 1 -- 172.21.15.16:0/1763860700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d0146a00 con 0x7f14d0155df0 2024-08-17T02:48:53.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.412+0000 7f14d5e00640 1 -- 172.21.15.16:0/1763860700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d0152780 con 0x7f14d0155df0 2024-08-17T02:48:53.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.413+0000 7f14c6000640 1 -- 172.21.15.16:0/1763860700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f14c8038050 con 0x7f14d0155df0 2024-08-17T02:48:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.413+0000 7f14c6000640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f14b0065180 0x7f14b0067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.413+0000 7f14c6000640 1 -- 172.21.15.16:0/1763860700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f14c80a2830 con 0x7f14d0155df0 2024-08-17T02:48:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.414+0000 7f14cea00640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f14b0065180 0x7f14b0067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.415+0000 7f14cea00640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f14b0065180 0x7f14b0067630 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14c0002860 tx=0x7f14c0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.417+0000 7f14c6000640 1 -- 172.21.15.16:0/1763860700 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f14d0152780 con 0x7f14d0155df0 2024-08-17T02:48:53.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.599+0000 7f14d5e00640 1 -- 172.21.15.16:0/1763860700 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f14d0064db0 con 0x7f14b0065180 2024-08-17T02:48:53.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.600+0000 7f14c6000640 1 -- 172.21.15.16:0/1763860700 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f14d0064db0 con 0x7f14b0065180 2024-08-17T02:48:53.601 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 1 to deep-scrub 2024-08-17T02:48:53.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.602+0000 7f14d5e00640 1 -- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f14b0065180 msgr2=0x7f14b0067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.602+0000 7f14d5e00640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f14b0065180 0x7f14b0067630 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14c0002860 tx=0x7f14c0009040 comp rx=0 tx=0).stop 2024-08-17T02:48:53.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.602+0000 7f14d5e00640 1 -- 172.21.15.16:0/1763860700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d0155df0 msgr2=0x7f14d014f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.602+0000 7f14d5e00640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d0155df0 0x7f14d014f080 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009f20 tx=0x7f14c8036000 comp rx=0 tx=0).stop 2024-08-17T02:48:53.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.602+0000 7f14d5e00640 1 -- 172.21.15.16:0/1763860700 shutdown_connections 2024-08-17T02:48:53.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.602+0000 7f14d5e00640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f14b0065180 0x7f14b0067630 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.602+0000 7f14d5e00640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14d0155df0 0x7f14d014f080 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.602+0000 7f14d5e00640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f14d0155a20 0x7f14d0151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.602+0000 7f14d5e00640 1 --2- 172.21.15.16:0/1763860700 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f14d006df00 0x7f14d01464c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.602+0000 7f14d5e00640 1 -- 172.21.15.16:0/1763860700 >> 172.21.15.16:0/1763860700 conn(0x7f14d00cb9b0 msgr2=0x7f14d0156b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:53.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.602+0000 7f14d5e00640 1 -- 172.21.15.16:0/1763860700 shutdown_connections 2024-08-17T02:48:53.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.602+0000 7f14d5e00640 1 -- 172.21.15.16:0/1763860700 wait complete. 2024-08-17T02:48:53.618 INFO:tasks.ceph:Scrubbing osd.2 2024-08-17T02:48:53.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:48:53.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.773+0000 7f746a800640 1 Processor -- start 2024-08-17T02:48:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.773+0000 7f746a800640 1 -- start start 2024-08-17T02:48:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f746a800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f746406e5f0 0x7f74640c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f746a800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7464156c60 0x7f74640c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f746a800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74640ae9b0 con 0x7f7464156890 2024-08-17T02:48:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f746a800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74640c8bb0 con 0x7f746406e5f0 2024-08-17T02:48:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f746a800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74640aa730 con 0x7f7464156c60 2024-08-17T02:48:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f745be00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f746406e5f0 0x7f74640c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f745be00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f746406e5f0 0x7f74640c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39764/0 (socket says 172.21.15.16:39764) 2024-08-17T02:48:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f745be00640 1 -- 172.21.15.16:0/4183367614 learned_addr learned my addr 172.21.15.16:0/4183367614 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f7463400640 1 --1- 172.21.15.16:0/4183367614 >> v1:172.21.15.16:6789/0 conn(0x7f7464156890 0x7f746406dee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35034/0 (socket says 172.21.15.16:35034) 2024-08-17T02:48:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f7463e00640 1 --2- 172.21.15.16:0/4183367614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7464156c60 0x7f74640c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f745be00640 1 -- 172.21.15.16:0/4183367614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7464156c60 msgr2=0x7f74640c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f745be00640 1 --2- 172.21.15.16:0/4183367614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7464156c60 0x7f74640c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f745be00640 1 -- 172.21.15.16:0/4183367614 >> v1:172.21.15.16:6789/0 conn(0x7f7464156890 legacy=0x7f746406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f745be00640 1 -- 172.21.15.16:0/4183367614 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74640ab320 con 0x7f746406e5f0 2024-08-17T02:48:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.775+0000 7f7462a00640 1 -- 172.21.15.16:0/4183367614 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745239090 0 0) 0x7f74640ae9b0 con 0x7f7464156890 2024-08-17T02:48:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.776+0000 7f745be00640 1 --2- 172.21.15.16:0/4183367614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f746406e5f0 0x7f74640c8130 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f744c009910 tx=0x7f744c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac772861541e4ec8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.777+0000 7f7462a00640 1 -- 172.21.15.16:0/4183367614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c02adb0 con 0x7f746406e5f0 2024-08-17T02:48:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.777+0000 7f7462a00640 1 -- 172.21.15.16:0/4183367614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744c032440 con 0x7f746406e5f0 2024-08-17T02:48:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.777+0000 7f7462a00640 1 -- 172.21.15.16:0/4183367614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744c0327e0 con 0x7f746406e5f0 2024-08-17T02:48:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.777+0000 7f746a800640 1 -- 172.21.15.16:0/4183367614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f746406e5f0 msgr2=0x7f74640c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.777+0000 7f746a800640 1 --2- 172.21.15.16:0/4183367614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f746406e5f0 0x7f74640c8130 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f744c009910 tx=0x7f744c0091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.777+0000 7f746a800640 1 -- 172.21.15.16:0/4183367614 shutdown_connections 2024-08-17T02:48:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.777+0000 7f746a800640 1 --2- 172.21.15.16:0/4183367614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f746406e5f0 0x7f74640c8130 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.777+0000 7f746a800640 1 --2- 172.21.15.16:0/4183367614 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7464156c60 0x7f74640c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.777+0000 7f746a800640 1 -- 172.21.15.16:0/4183367614 >> 172.21.15.16:0/4183367614 conn(0x7f7464067750 msgr2=0x7f746406c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.777+0000 7f746a800640 1 -- 172.21.15.16:0/4183367614 shutdown_connections 2024-08-17T02:48:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.777+0000 7f746a800640 1 -- 172.21.15.16:0/4183367614 wait complete. 2024-08-17T02:48:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.779+0000 7f746a800640 1 Processor -- start 2024-08-17T02:48:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.779+0000 7f746a800640 1 -- start start 2024-08-17T02:48:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f746a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746406e5f0 0x7f746413a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f746a800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7464156890 0x7f746413bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f746a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464156c60 0x7f746413c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f746a800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74641534b0 con 0x7f7464156c60 2024-08-17T02:48:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f746a800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746415c3c0 con 0x7f746406e5f0 2024-08-17T02:48:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f746a800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746413c5b0 con 0x7f7464156890 2024-08-17T02:48:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f7463e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464156c60 0x7f746413c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f7463e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464156c60 0x7f746413c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39780/0 (socket says 172.21.15.16:39780) 2024-08-17T02:48:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f7463e00640 1 -- 172.21.15.16:0/3275081905 learned_addr learned my addr 172.21.15.16:0/3275081905 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f7463e00640 1 -- 172.21.15.16:0/3275081905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7464156890 msgr2=0x7f746413bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f7463e00640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7464156890 0x7f746413bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f7463e00640 1 -- 172.21.15.16:0/3275081905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746406e5f0 msgr2=0x7f746413a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f7463400640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746406e5f0 0x7f746413a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f7463e00640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746406e5f0 0x7f746413a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f7463e00640 1 -- 172.21.15.16:0/3275081905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c009450 con 0x7f7464156c60 2024-08-17T02:48:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.780+0000 7f7463400640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f746406e5f0 0x7f746413a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-17T02:48:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.781+0000 7f7463e00640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464156c60 0x7f746413c070 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f745c009fd0 tx=0x7f745c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.781+0000 7f745aa00640 1 -- 172.21.15.16:0/3275081905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c00ee30 con 0x7f7464156c60 2024-08-17T02:48:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.781+0000 7f745aa00640 1 -- 172.21.15.16:0/3275081905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c014420 con 0x7f7464156c60 2024-08-17T02:48:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.781+0000 7f745aa00640 1 -- 172.21.15.16:0/3275081905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c0147e0 con 0x7f7464156c60 2024-08-17T02:48:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.781+0000 7f746a800640 1 -- 172.21.15.16:0/3275081905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746415c090 con 0x7f7464156c60 2024-08-17T02:48:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.781+0000 7f746a800640 1 -- 172.21.15.16:0/3275081905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74641531d0 con 0x7f7464156c60 2024-08-17T02:48:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.782+0000 7f746a800640 1 -- 172.21.15.16:0/3275081905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f74640710c0 con 0x7f7464156c60 2024-08-17T02:48:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.783+0000 7f745aa00640 1 -- 172.21.15.16:0/3275081905 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f745c014b50 con 0x7f7464156c60 2024-08-17T02:48:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.784+0000 7f745aa00640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7440065150 0x7f7440067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.784+0000 7f745aa00640 1 -- 172.21.15.16:0/3275081905 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f745c0829b0 con 0x7f7464156c60 2024-08-17T02:48:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.784+0000 7f745aa00640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] conn(0x7f744006b040 0x7f744006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.784+0000 7f745aa00640 1 -- 172.21.15.16:0/3275081905 --> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f744006db00 con 0x7f744006b040 2024-08-17T02:48:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.784+0000 7f745aa00640 1 -- 172.21.15.16:0/3275081905 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f745c087240 con 0x7f7464156c60 2024-08-17T02:48:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.784+0000 7f7463400640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7440065150 0x7f7440067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.784+0000 7f745be00640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] conn(0x7f744006b040 0x7f744006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.785+0000 7f745be00640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] conn(0x7f744006b040 0x7f744006d450 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.785+0000 7f7463400640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7440065150 0x7f7440067600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7454001840 tx=0x7f7454009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.786+0000 7f745aa00640 1 -- 172.21.15.16:0/3275081905 <== osd.2 v2:172.21.15.16:6816/37183 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f744006db00 con 0x7f744006b040 2024-08-17T02:48:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.798+0000 7f746a800640 1 -- 172.21.15.16:0/3275081905 --> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f746414abc0 con 0x7f744006b040 2024-08-17T02:48:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.799+0000 7f745aa00640 1 -- 172.21.15.16:0/3275081905 <== osd.2 v2:172.21.15.16:6816/37183 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f746414abc0 con 0x7f744006b040 2024-08-17T02:48:53.800 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-17T02:48:53.800 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:48:53.800 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-17T02:48:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.799+0000 7f746a800640 1 -- 172.21.15.16:0/3275081905 >> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] conn(0x7f744006b040 msgr2=0x7f744006d450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.799+0000 7f746a800640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.16:6816/37183,v1:172.21.15.16:6817/37183] conn(0x7f744006b040 0x7f744006d450 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.799+0000 7f746a800640 1 -- 172.21.15.16:0/3275081905 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7440065150 msgr2=0x7f7440067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.799+0000 7f746a800640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f7440065150 0x7f7440067600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7454001840 tx=0x7f7454009040 comp rx=0 tx=0).stop 2024-08-17T02:48:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.799+0000 7f746a800640 1 -- 172.21.15.16:0/3275081905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464156c60 msgr2=0x7f746413c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.799+0000 7f746a800640 1 --2- 172.21.15.16:0/3275081905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7464156c60 0x7f746413c070 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f745c009fd0 tx=0x7f745c009900 comp rx=0 tx=0).stop 2024-08-17T02:48:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.799+0000 7f7463e00640 1 -- 172.21.15.16:0/3275081905 reap_dead start 2024-08-17T02:48:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.799+0000 7f746a800640 1 -- 172.21.15.16:0/3275081905 shutdown_connections 2024-08-17T02:48:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.799+0000 7f746a800640 1 -- 172.21.15.16:0/3275081905 >> 172.21.15.16:0/3275081905 conn(0x7f7464067750 msgr2=0x7f74641551f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.800+0000 7f746a800640 1 -- 172.21.15.16:0/3275081905 shutdown_connections 2024-08-17T02:48:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.800+0000 7f746a800640 1 -- 172.21.15.16:0/3275081905 wait complete. 2024-08-17T02:48:53.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-08-17T02:48:53.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.965+0000 7fc1a1000640 1 Processor -- start 2024-08-17T02:48:53.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.965+0000 7fc1a1000640 1 -- start start 2024-08-17T02:48:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.967+0000 7fc1a1000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c155a20 0x7fc19c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.967+0000 7fc1a1000640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19c156670 con 0x7fc19c06df00 2024-08-17T02:48:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.967+0000 7fc1a1000640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19c06e5e0 con 0x7fc19c155df0 2024-08-17T02:48:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.967+0000 7fc1a1000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0b0350 con 0x7fc19c155a20 2024-08-17T02:48:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.967+0000 7fc19a800640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc19c06df00 0x7fc19c156b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34764/0 (socket says 172.21.15.16:34764) 2024-08-17T02:48:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.967+0000 7fc19a800640 1 -- 172.21.15.16:0/1254976133 learned_addr learned my addr 172.21.15.16:0/1254976133 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.967+0000 7fc199e00640 1 --2- 172.21.15.16:0/1254976133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c155a20 0x7fc19c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.967+0000 7fc199400640 1 -- 172.21.15.16:0/1254976133 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129324827 0 0) 0x7fc19c156670 con 0x7fc19c06df00 2024-08-17T02:48:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.967+0000 7fc199400640 1 -- 172.21.15.16:0/1254976133 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1800034e0 con 0x7fc19c06df00 2024-08-17T02:48:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.968+0000 7fc199400640 1 -- 172.21.15.16:0/1254976133 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133986536 0 0) 0x7fc19c06e5e0 con 0x7fc19c155df0 2024-08-17T02:48:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.968+0000 7fc199400640 1 -- 172.21.15.16:0/1254976133 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc19c156670 con 0x7fc19c155df0 2024-08-17T02:48:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.968+0000 7fc199e00640 1 -- 172.21.15.16:0/1254976133 >> v1:172.21.15.142:6789/0 conn(0x7fc19c155df0 legacy=0x7fc19c06d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.968+0000 7fc199e00640 1 -- 172.21.15.16:0/1254976133 >> v1:172.21.15.16:6789/0 conn(0x7fc19c06df00 legacy=0x7fc19c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.968+0000 7fc199400640 1 -- 172.21.15.16:0/1254976133 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4244363139 0 0) 0x7fc1800034e0 con 0x7fc19c06df00 2024-08-17T02:48:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.968+0000 7fc199e00640 1 -- 172.21.15.16:0/1254976133 --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c06e8c0 con 0x7fc19c155a20 2024-08-17T02:48:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.969+0000 7fc199e00640 1 --2- 172.21.15.16:0/1254976133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c155a20 0x7fc19c0adf10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009880 tx=0x7fc18c0090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b082d6e5f9cfa69 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.969+0000 7fc199400640 1 -- 172.21.15.16:0/1254976133 <== mon.2 v2:172.21.15.142:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1940017f0 con 0x7fc19c155a20 2024-08-17T02:48:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.969+0000 7fc199400640 1 -- 172.21.15.16:0/1254976133 <== mon.2 v2:172.21.15.142:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c02aac0 con 0x7fc19c155a20 2024-08-17T02:48:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.969+0000 7fc199400640 1 -- 172.21.15.16:0/1254976133 <== mon.2 v2:172.21.15.142:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c02ae60 con 0x7fc19c155a20 2024-08-17T02:48:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.970+0000 7fc1a1000640 1 -- 172.21.15.16:0/1254976133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c155a20 msgr2=0x7fc19c0adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.970+0000 7fc1a1000640 1 --2- 172.21.15.16:0/1254976133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c155a20 0x7fc19c0adf10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009880 tx=0x7fc18c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.970+0000 7fc1a1000640 1 -- 172.21.15.16:0/1254976133 shutdown_connections 2024-08-17T02:48:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.970+0000 7fc1a1000640 1 --2- 172.21.15.16:0/1254976133 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c155a20 0x7fc19c0adf10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.970+0000 7fc1a1000640 1 -- 172.21.15.16:0/1254976133 >> 172.21.15.16:0/1254976133 conn(0x7fc19c0cb9b0 msgr2=0x7fc19c0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.970+0000 7fc1a1000640 1 -- 172.21.15.16:0/1254976133 shutdown_connections 2024-08-17T02:48:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.970+0000 7fc1a1000640 1 -- 172.21.15.16:0/1254976133 wait complete. 2024-08-17T02:48:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.971+0000 7fc1a1000640 1 Processor -- start 2024-08-17T02:48:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.971+0000 7fc1a1000640 1 -- start start 2024-08-17T02:48:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.972+0000 7fc1a1000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc19c06df00 0x7fc19c0ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.972+0000 7fc1a1000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c155a20 0x7fc19c150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.972+0000 7fc1a1000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c155df0 0x7fc19c0aa020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.972+0000 7fc1a1000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c1513a0 con 0x7fc19c155df0 2024-08-17T02:48:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.972+0000 7fc1a1000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c168e50 con 0x7fc19c06df00 2024-08-17T02:48:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.972+0000 7fc1a1000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0aa560 con 0x7fc19c155a20 2024-08-17T02:48:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.972+0000 7fc19a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c155df0 0x7fc19c0aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.973+0000 7fc19a800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c155df0 0x7fc19c0aa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36606/0 (socket says 172.21.15.16:36606) 2024-08-17T02:48:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.973+0000 7fc19a800640 1 -- 172.21.15.16:0/3793505498 learned_addr learned my addr 172.21.15.16:0/3793505498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.973+0000 7fc199e00640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc19c06df00 0x7fc19c0ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.973+0000 7fc191400640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c155a20 0x7fc19c150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.973+0000 7fc19a800640 1 -- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c155a20 msgr2=0x7fc19c150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.973+0000 7fc19a800640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c155a20 0x7fc19c150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.973+0000 7fc19a800640 1 -- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc19c06df00 msgr2=0x7fc19c0ac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.973+0000 7fc19a800640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc19c06df00 0x7fc19c0ac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.973+0000 7fc19a800640 1 -- 172.21.15.16:0/3793505498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c009390 con 0x7fc19c155df0 2024-08-17T02:48:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.973+0000 7fc19a800640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c155df0 0x7fc19c0aa020 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc194001920 tx=0x7fc194007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.973+0000 7fc192000640 1 -- 172.21.15.16:0/3793505498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194004b20 con 0x7fc19c155df0 2024-08-17T02:48:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.974+0000 7fc192000640 1 -- 172.21.15.16:0/3793505498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc194004ca0 con 0x7fc19c155df0 2024-08-17T02:48:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.974+0000 7fc1a1000640 1 -- 172.21.15.16:0/3793505498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19c0aa7f0 con 0x7fc19c155df0 2024-08-17T02:48:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.974+0000 7fc192000640 1 -- 172.21.15.16:0/3793505498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19400acf0 con 0x7fc19c155df0 2024-08-17T02:48:53.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.974+0000 7fc1a1000640 1 -- 172.21.15.16:0/3793505498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c0aadb0 con 0x7fc19c155df0 2024-08-17T02:48:53.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.975+0000 7fc1a1000640 1 -- 172.21.15.16:0/3793505498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19c168fc0 con 0x7fc19c155df0 2024-08-17T02:48:53.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.976+0000 7fc192000640 1 -- 172.21.15.16:0/3793505498 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fc194007360 con 0x7fc19c155df0 2024-08-17T02:48:53.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.976+0000 7fc192000640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc178065150 0x7fc178067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:53.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.976+0000 7fc192000640 1 -- 172.21.15.16:0/3793505498 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fc194081cc0 con 0x7fc19c155df0 2024-08-17T02:48:53.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.977+0000 7fc199e00640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc178065150 0x7fc178067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:53.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.977+0000 7fc199e00640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc178065150 0x7fc178067600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc18c034000 tx=0x7fc18c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:53.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:53.980+0000 7fc192000640 1 -- 172.21.15.16:0/3793505498 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fc194088080 con 0x7fc19c155df0 2024-08-17T02:48:54.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.162+0000 7fc1a1000640 1 -- 172.21.15.16:0/3793505498 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc19c064db0 con 0x7fc178065150 2024-08-17T02:48:54.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.162+0000 7fc192000640 1 -- 172.21.15.16:0/3793505498 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc19c064db0 con 0x7fc178065150 2024-08-17T02:48:54.164 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 2 to deep-scrub 2024-08-17T02:48:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.165+0000 7fc1a1000640 1 -- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc178065150 msgr2=0x7fc178067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.165+0000 7fc1a1000640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc178065150 0x7fc178067600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc18c034000 tx=0x7fc18c032000 comp rx=0 tx=0).stop 2024-08-17T02:48:54.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.165+0000 7fc1a1000640 1 -- 172.21.15.16:0/3793505498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c155df0 msgr2=0x7fc19c0aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.165+0000 7fc1a1000640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c155df0 0x7fc19c0aa020 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc194001920 tx=0x7fc194007d10 comp rx=0 tx=0).stop 2024-08-17T02:48:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.165+0000 7fc1a1000640 1 -- 172.21.15.16:0/3793505498 shutdown_connections 2024-08-17T02:48:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.165+0000 7fc1a1000640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fc178065150 0x7fc178067600 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.165+0000 7fc1a1000640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc19c155df0 0x7fc19c0aa020 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.165+0000 7fc1a1000640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fc19c155a20 0x7fc19c150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.165+0000 7fc1a1000640 1 --2- 172.21.15.16:0/3793505498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc19c06df00 0x7fc19c0ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.165+0000 7fc1a1000640 1 -- 172.21.15.16:0/3793505498 >> 172.21.15.16:0/3793505498 conn(0x7fc19c0cb9b0 msgr2=0x7fc19c1526e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.165+0000 7fc1a1000640 1 -- 172.21.15.16:0/3793505498 shutdown_connections 2024-08-17T02:48:54.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.165+0000 7fc1a1000640 1 -- 172.21.15.16:0/3793505498 wait complete. 2024-08-17T02:48:54.181 INFO:tasks.ceph:Scrubbing osd.3 2024-08-17T02:48:54.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:48:54.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.335+0000 7f9401400640 1 Processor -- start 2024-08-17T02:48:54.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.335+0000 7f9401400640 1 -- start start 2024-08-17T02:48:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.336+0000 7f9401400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc150130 0x7f93fc0aa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.336+0000 7f9401400640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93fc06e5f0 0x7f93fc0ab510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.336+0000 7f9401400640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93fc1568b0 con 0x7f93fc156d40 2024-08-17T02:48:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.336+0000 7f9401400640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93fc150500 con 0x7f93fc06e5f0 2024-08-17T02:48:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.336+0000 7f9401400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc155fc0 con 0x7f93fc150130 2024-08-17T02:48:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.337+0000 7f93fb200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93fc06e5f0 0x7f93fc0ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.337+0000 7f93fb200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93fc06e5f0 0x7f93fc0ab510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36608/0 (socket says 172.21.15.16:36608) 2024-08-17T02:48:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.337+0000 7f93fb200640 1 -- 172.21.15.16:0/2871310250 learned_addr learned my addr 172.21.15.16:0/2871310250 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.337+0000 7f93fa800640 1 --2- 172.21.15.16:0/2871310250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc150130 0x7f93fc0aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.337+0000 7f93fb200640 1 -- 172.21.15.16:0/2871310250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc150130 msgr2=0x7f93fc0aa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.337+0000 7f93fb200640 1 --2- 172.21.15.16:0/2871310250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc150130 0x7f93fc0aa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.337+0000 7f93fb200640 1 -- 172.21.15.16:0/2871310250 >> v1:172.21.15.142:6789/0 conn(0x7f93fc156d40 legacy=0x7f93fc06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.337+0000 7f93fb200640 1 -- 172.21.15.16:0/2871310250 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93fc0add60 con 0x7f93fc06e5f0 2024-08-17T02:48:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.337+0000 7f93fa800640 1 --2- 172.21.15.16:0/2871310250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc150130 0x7f93fc0aa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.338+0000 7f93fb200640 1 --2- 172.21.15.16:0/2871310250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc06e5f0 0x7f93fc0ab510 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009f20 tx=0x7f93f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4427c9a40fb081ea server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.338+0000 7f93f9e00640 1 -- 172.21.15.16:0/2871310250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4031400 con 0x7f93fc06e5f0 2024-08-17T02:48:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.338+0000 7f93f9e00640 1 -- 172.21.15.16:0/2871310250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f402d050 con 0x7f93fc06e5f0 2024-08-17T02:48:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.338+0000 7f93f9e00640 1 -- 172.21.15.16:0/2871310250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f40317d0 con 0x7f93fc06e5f0 2024-08-17T02:48:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.339+0000 7f9401400640 1 -- 172.21.15.16:0/2871310250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc06e5f0 msgr2=0x7f93fc0ab510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.339+0000 7f9401400640 1 --2- 172.21.15.16:0/2871310250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc06e5f0 0x7f93fc0ab510 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009f20 tx=0x7f93f4009840 comp rx=0 tx=0).stop 2024-08-17T02:48:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.339+0000 7f9401400640 1 -- 172.21.15.16:0/2871310250 shutdown_connections 2024-08-17T02:48:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.339+0000 7f9401400640 1 --2- 172.21.15.16:0/2871310250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc06e5f0 0x7f93fc0ab510 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.339+0000 7f9401400640 1 --2- 172.21.15.16:0/2871310250 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc150130 0x7f93fc0aa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.339+0000 7f9401400640 1 -- 172.21.15.16:0/2871310250 >> 172.21.15.16:0/2871310250 conn(0x7f93fc067750 msgr2=0x7f93fc06c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.339+0000 7f9401400640 1 -- 172.21.15.16:0/2871310250 shutdown_connections 2024-08-17T02:48:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.339+0000 7f9401400640 1 -- 172.21.15.16:0/2871310250 wait complete. 2024-08-17T02:48:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.340+0000 7f9401400640 1 Processor -- start 2024-08-17T02:48:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.341+0000 7f9401400640 1 -- start start 2024-08-17T02:48:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.341+0000 7f9401400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93fc06e5f0 0x7f93fc0a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f9401400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc150130 0x7f93fc13b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f9401400640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc156d40 0x7f93fc0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f9401400640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0a8370 con 0x7f93fc150130 2024-08-17T02:48:54.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f9401400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc0a79a0 con 0x7f93fc06e5f0 2024-08-17T02:48:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f9401400640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93fc13bbf0 con 0x7f93fc156d40 2024-08-17T02:48:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f93f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc150130 0x7f93fc13b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f93f3e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc150130 0x7f93fc13b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36610/0 (socket says 172.21.15.16:36610) 2024-08-17T02:48:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f93f3e00640 1 -- 172.21.15.16:0/3112007753 learned_addr learned my addr 172.21.15.16:0/3112007753 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f93fa800640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93fc06e5f0 0x7f93fc0a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f93fb200640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc156d40 0x7f93fc0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f93f3e00640 1 -- 172.21.15.16:0/3112007753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc156d40 msgr2=0x7f93fc0a63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f93f3e00640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc156d40 0x7f93fc0a63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f93f3e00640 1 -- 172.21.15.16:0/3112007753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93fc06e5f0 msgr2=0x7f93fc0a7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f93f3e00640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93fc06e5f0 0x7f93fc0a7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f93f3e00640 1 -- 172.21.15.16:0/3112007753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f4009ae0 con 0x7f93fc150130 2024-08-17T02:48:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.342+0000 7f93fb200640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f93fc156d40 0x7f93fc0a63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.343+0000 7f93f3e00640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc150130 0x7f93fc13b6b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f93e400c4c0 tx=0x7f93e4007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.343+0000 7f93f2000640 1 -- 172.21.15.16:0/3112007753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4011590 con 0x7f93fc150130 2024-08-17T02:48:54.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.343+0000 7f93f2000640 1 -- 172.21.15.16:0/3112007753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e4011710 con 0x7f93fc150130 2024-08-17T02:48:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.343+0000 7f93f2000640 1 -- 172.21.15.16:0/3112007753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e4011ad0 con 0x7f93fc150130 2024-08-17T02:48:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.343+0000 7f9401400640 1 -- 172.21.15.16:0/3112007753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93fc0a6d90 con 0x7f93fc150130 2024-08-17T02:48:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.343+0000 7f9401400640 1 -- 172.21.15.16:0/3112007753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93fc0a7350 con 0x7f93fc150130 2024-08-17T02:48:54.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.344+0000 7f9401400640 1 -- 172.21.15.16:0/3112007753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f93fc0710c0 con 0x7f93fc150130 2024-08-17T02:48:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.345+0000 7f93f2000640 1 -- 172.21.15.16:0/3112007753 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f93e401a360 con 0x7f93fc150130 2024-08-17T02:48:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.346+0000 7f93f2000640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f93d8065150 0x7f93d8067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.346+0000 7f93f2000640 1 -- 172.21.15.16:0/3112007753 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f93e4081100 con 0x7f93fc150130 2024-08-17T02:48:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.346+0000 7f93f2000640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] conn(0x7f93d806b040 0x7f93d806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.346+0000 7f93f2000640 1 -- 172.21.15.16:0/3112007753 --> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93d806db00 con 0x7f93d806b040 2024-08-17T02:48:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.346+0000 7f93f2000640 1 -- 172.21.15.16:0/3112007753 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93e4083740 con 0x7f93fc150130 2024-08-17T02:48:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.346+0000 7f93fb200640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] conn(0x7f93d806b040 0x7f93d806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.346+0000 7f93fa800640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f93d8065150 0x7f93d8067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.347+0000 7f93fb200640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] conn(0x7f93d806b040 0x7f93d806d450 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:54.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.347+0000 7f93fa800640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f93d8065150 0x7f93d8067600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f93ec006e70 tx=0x7f93ec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:54.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.347+0000 7f93f2000640 1 -- 172.21.15.16:0/3112007753 <== osd.3 v2:172.21.15.16:6808/37181 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f93d806db00 con 0x7f93d806b040 2024-08-17T02:48:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.360+0000 7f9401400640 1 -- 172.21.15.16:0/3112007753 --> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f93fc0aa980 con 0x7f93d806b040 2024-08-17T02:48:54.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.360+0000 7f93f2000640 1 -- 172.21.15.16:0/3112007753 <== osd.3 v2:172.21.15.16:6808/37181 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f93fc0aa980 con 0x7f93d806b040 2024-08-17T02:48:54.362 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-17T02:48:54.362 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:48:54.362 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-17T02:48:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.361+0000 7f9401400640 1 -- 172.21.15.16:0/3112007753 >> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] conn(0x7f93d806b040 msgr2=0x7f93d806d450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.361+0000 7f9401400640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.16:6808/37181,v1:172.21.15.16:6809/37181] conn(0x7f93d806b040 0x7f93d806d450 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.361+0000 7f9401400640 1 -- 172.21.15.16:0/3112007753 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f93d8065150 msgr2=0x7f93d8067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.361+0000 7f9401400640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f93d8065150 0x7f93d8067600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f93ec006e70 tx=0x7f93ec008040 comp rx=0 tx=0).stop 2024-08-17T02:48:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.361+0000 7f9401400640 1 -- 172.21.15.16:0/3112007753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc150130 msgr2=0x7f93fc13b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.361+0000 7f9401400640 1 --2- 172.21.15.16:0/3112007753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93fc150130 0x7f93fc13b6b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f93e400c4c0 tx=0x7f93e4007ec0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.361+0000 7f93fb200640 1 -- 172.21.15.16:0/3112007753 reap_dead start 2024-08-17T02:48:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.361+0000 7f9401400640 1 -- 172.21.15.16:0/3112007753 shutdown_connections 2024-08-17T02:48:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.361+0000 7f9401400640 1 -- 172.21.15.16:0/3112007753 >> 172.21.15.16:0/3112007753 conn(0x7f93fc067750 msgr2=0x7f93fc069100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.361+0000 7f9401400640 1 -- 172.21.15.16:0/3112007753 shutdown_connections 2024-08-17T02:48:54.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.361+0000 7f9401400640 1 -- 172.21.15.16:0/3112007753 wait complete. 2024-08-17T02:48:54.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-08-17T02:48:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.526+0000 7f3b9fc00640 1 Processor -- start 2024-08-17T02:48:54.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.526+0000 7f3b9fc00640 1 -- start start 2024-08-17T02:48:54.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.527+0000 7f3b9fc00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f3b98155180 0x7f3b980aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.527+0000 7f3b9fc00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b980afc10 con 0x7f3b98155550 2024-08-17T02:48:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.527+0000 7f3b9fc00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b980af000 con 0x7f3b9806e080 2024-08-17T02:48:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.527+0000 7f3b9fc00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b980af2e0 con 0x7f3b98155180 2024-08-17T02:48:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.528+0000 7f3b97e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3b98155550 0x7f3b980ae070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34774/0 (socket says 172.21.15.16:34774) 2024-08-17T02:48:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.528+0000 7f3b97e00640 1 -- 172.21.15.16:0/1537556021 learned_addr learned my addr 172.21.15.16:0/1537556021 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.528+0000 7f3b9d200640 1 --2- 172.21.15.16:0/1537556021 >> v2:172.21.15.142:3300/0 conn(0x7f3b98155180 0x7f3b980aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.528+0000 7f3b97400640 1 -- 172.21.15.16:0/1537556021 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821407116 0 0) 0x7f3b980afc10 con 0x7f3b98155550 2024-08-17T02:48:54.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.528+0000 7f3b97400640 1 -- 172.21.15.16:0/1537556021 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b800045f0 con 0x7f3b98155550 2024-08-17T02:48:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.528+0000 7f3b9d200640 1 -- 172.21.15.16:0/1537556021 >> v1:172.21.15.142:6789/0 conn(0x7f3b9806e080 legacy=0x7f3b9806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.528+0000 7f3b9d200640 1 -- 172.21.15.16:0/1537556021 >> v1:172.21.15.16:6789/0 conn(0x7f3b98155550 legacy=0x7f3b980ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.528+0000 7f3b9d200640 1 -- 172.21.15.16:0/1537556021 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b980af5c0 con 0x7f3b98155180 2024-08-17T02:48:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.529+0000 7f3b9d200640 1 --2- 172.21.15.16:0/1537556021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b98155180 0x7f3b980aea20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009850 tx=0x7f3b8c0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=966e70e97fb685d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.530+0000 7f3b97400640 1 -- 172.21.15.16:0/1537556021 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c02ae20 con 0x7f3b98155180 2024-08-17T02:48:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.530+0000 7f3b97400640 1 -- 172.21.15.16:0/1537556021 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8c032440 con 0x7f3b98155180 2024-08-17T02:48:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.530+0000 7f3b97400640 1 -- 172.21.15.16:0/1537556021 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0327c0 con 0x7f3b98155180 2024-08-17T02:48:54.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.530+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1537556021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b98155180 msgr2=0x7f3b980aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.530+0000 7f3b9fc00640 1 --2- 172.21.15.16:0/1537556021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b98155180 0x7f3b980aea20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009850 tx=0x7f3b8c0090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.530+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1537556021 shutdown_connections 2024-08-17T02:48:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.530+0000 7f3b9fc00640 1 --2- 172.21.15.16:0/1537556021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b98155180 0x7f3b980aea20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.530+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1537556021 >> 172.21.15.16:0/1537556021 conn(0x7f3b980cb9b0 msgr2=0x7f3b980cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.530+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1537556021 shutdown_connections 2024-08-17T02:48:54.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.531+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1537556021 wait complete. 2024-08-17T02:48:54.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.532+0000 7f3b9fc00640 1 Processor -- start 2024-08-17T02:48:54.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.532+0000 7f3b9fc00640 1 -- start start 2024-08-17T02:48:54.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.533+0000 7f3b9fc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9806e080 0x7f3b981483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.533+0000 7f3b9fc00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b98155180 0x7f3b980c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.533+0000 7f3b9fc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b98155550 0x7f3b980c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.533+0000 7f3b9fc00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b98157f40 con 0x7f3b98155550 2024-08-17T02:48:54.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.533+0000 7f3b9fc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b980c25c0 con 0x7f3b9806e080 2024-08-17T02:48:54.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.533+0000 7f3b9fc00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b98157610 con 0x7f3b98155180 2024-08-17T02:48:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.533+0000 7f3b9dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b98155550 0x7f3b980c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.533+0000 7f3b9dc00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b98155550 0x7f3b980c1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36622/0 (socket says 172.21.15.16:36622) 2024-08-17T02:48:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.533+0000 7f3b9dc00640 1 -- 172.21.15.16:0/1852401502 learned_addr learned my addr 172.21.15.16:0/1852401502 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.533+0000 7f3b9d200640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9806e080 0x7f3b981483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.533+0000 7f3b97e00640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b98155180 0x7f3b980c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.534+0000 7f3b9dc00640 1 -- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b98155180 msgr2=0x7f3b980c1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.534+0000 7f3b9dc00640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b98155180 0x7f3b980c1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.534+0000 7f3b9dc00640 1 -- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9806e080 msgr2=0x7f3b981483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.534+0000 7f3b9dc00640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9806e080 0x7f3b981483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.534+0000 7f3b9dc00640 1 -- 172.21.15.16:0/1852401502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c009390 con 0x7f3b98155550 2024-08-17T02:48:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.534+0000 7f3b9dc00640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b98155550 0x7f3b980c1f60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000c860 tx=0x7f3b90001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.534+0000 7f3b8be00640 1 -- 172.21.15.16:0/1852401502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9000ce50 con 0x7f3b98155550 2024-08-17T02:48:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.534+0000 7f3b8be00640 1 -- 172.21.15.16:0/1852401502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9000a850 con 0x7f3b98155550 2024-08-17T02:48:54.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.534+0000 7f3b8be00640 1 -- 172.21.15.16:0/1852401502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9000ac10 con 0x7f3b98155550 2024-08-17T02:48:54.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.534+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1852401502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b981578f0 con 0x7f3b98155550 2024-08-17T02:48:54.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.534+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1852401502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b981568c0 con 0x7f3b98155550 2024-08-17T02:48:54.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.535+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1852401502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b981580b0 con 0x7f3b98155550 2024-08-17T02:48:54.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.536+0000 7f3b8be00640 1 -- 172.21.15.16:0/1852401502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f3b90012630 con 0x7f3b98155550 2024-08-17T02:48:54.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.537+0000 7f3b8be00640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3b78065150 0x7f3b78067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.537+0000 7f3b8be00640 1 -- 172.21.15.16:0/1852401502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f3b900822a0 con 0x7f3b98155550 2024-08-17T02:48:54.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.537+0000 7f3b9d200640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3b78065150 0x7f3b78067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.538+0000 7f3b9d200640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3b78065150 0x7f3b78067600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c034000 tx=0x7f3b8c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:54.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.541+0000 7f3b8be00640 1 -- 172.21.15.16:0/1852401502 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f3b900506b0 con 0x7f3b98155550 2024-08-17T02:48:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.724+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1852401502 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b98064db0 con 0x7f3b78065150 2024-08-17T02:48:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.724+0000 7f3b8be00640 1 -- 172.21.15.16:0/1852401502 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3b98064db0 con 0x7f3b78065150 2024-08-17T02:48:54.725 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 3 to deep-scrub 2024-08-17T02:48:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.726+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3b78065150 msgr2=0x7f3b78067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.726+0000 7f3b9fc00640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3b78065150 0x7f3b78067600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c034000 tx=0x7f3b8c032000 comp rx=0 tx=0).stop 2024-08-17T02:48:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.726+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1852401502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b98155550 msgr2=0x7f3b980c1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.726+0000 7f3b9fc00640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b98155550 0x7f3b980c1f60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000c860 tx=0x7f3b90001690 comp rx=0 tx=0).stop 2024-08-17T02:48:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.726+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1852401502 shutdown_connections 2024-08-17T02:48:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.726+0000 7f3b9fc00640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f3b78065150 0x7f3b78067600 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.726+0000 7f3b9fc00640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b98155550 0x7f3b980c1f60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.726+0000 7f3b9fc00640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f3b98155180 0x7f3b980c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.726+0000 7f3b9fc00640 1 --2- 172.21.15.16:0/1852401502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b9806e080 0x7f3b981483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.726+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1852401502 >> 172.21.15.16:0/1852401502 conn(0x7f3b980cb9b0 msgr2=0x7f3b980ca8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.726+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1852401502 shutdown_connections 2024-08-17T02:48:54.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.726+0000 7f3b9fc00640 1 -- 172.21.15.16:0/1852401502 wait complete. 2024-08-17T02:48:54.743 INFO:tasks.ceph:Scrubbing osd.4 2024-08-17T02:48:54.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:48:54.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.898+0000 7f9008e00640 1 Processor -- start 2024-08-17T02:48:54.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.898+0000 7f9008e00640 1 -- start start 2024-08-17T02:48:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.899+0000 7f9008e00640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f9004155c50 0x7f90040af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.899+0000 7f9008e00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9004157840 con 0x7f90040af630 2024-08-17T02:48:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.899+0000 7f9008e00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90040afa70 con 0x7f9004155880 2024-08-17T02:48:54.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.899+0000 7f9008e00640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90040aed40 con 0x7f9004155c50 2024-08-17T02:48:54.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.899+0000 7f9003e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f90040af630 0x7f9004157cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34788/0 (socket says 172.21.15.16:34788) 2024-08-17T02:48:54.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.899+0000 7f9003e00640 1 -- 172.21.15.16:0/3906710020 learned_addr learned my addr 172.21.15.16:0/3906710020 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:54.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.899+0000 7f8ffbe00640 1 --2- 172.21.15.16:0/3906710020 >> v2:172.21.15.142:3300/0 conn(0x7f9004155c50 0x7f90040af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.900+0000 7f9002a00640 1 -- 172.21.15.16:0/3906710020 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273435190 0 0) 0x7f9004157840 con 0x7f90040af630 2024-08-17T02:48:54.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.900+0000 7f9002a00640 1 -- 172.21.15.16:0/3906710020 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fe80034e0 con 0x7f90040af630 2024-08-17T02:48:54.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.900+0000 7f9002a00640 1 -- 172.21.15.16:0/3906710020 <== mon.1 v1:172.21.15.142:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760777205 0 0) 0x7f90040afa70 con 0x7f9004155880 2024-08-17T02:48:54.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.900+0000 7f9002a00640 1 -- 172.21.15.16:0/3906710020 --> v1:172.21.15.142:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9004157840 con 0x7f9004155880 2024-08-17T02:48:54.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.900+0000 7f8ffbe00640 1 -- 172.21.15.16:0/3906710020 >> v1:172.21.15.142:6789/0 conn(0x7f9004155880 legacy=0x7f90040ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.900+0000 7f8ffbe00640 1 -- 172.21.15.16:0/3906710020 >> v1:172.21.15.16:6789/0 conn(0x7f90040af630 legacy=0x7f9004157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.900+0000 7f8ffbe00640 1 -- 172.21.15.16:0/3906710020 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9004156c30 con 0x7f9004155c50 2024-08-17T02:48:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.901+0000 7f8ffbe00640 1 --2- 172.21.15.16:0/3906710020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9004155c50 0x7f90040af0f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009850 tx=0x7f8fec0090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c744fdf3e80e122 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.902+0000 7f9002a00640 1 -- 172.21.15.16:0/3906710020 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec02ae20 con 0x7f9004155c50 2024-08-17T02:48:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.902+0000 7f9002a00640 1 -- 172.21.15.16:0/3906710020 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec032440 con 0x7f9004155c50 2024-08-17T02:48:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.902+0000 7f9002a00640 1 -- 172.21.15.16:0/3906710020 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec0327e0 con 0x7f9004155c50 2024-08-17T02:48:54.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.902+0000 7f9008e00640 1 -- 172.21.15.16:0/3906710020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9004155c50 msgr2=0x7f90040af0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.902+0000 7f9008e00640 1 --2- 172.21.15.16:0/3906710020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9004155c50 0x7f90040af0f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009850 tx=0x7f8fec0090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.902+0000 7f9008e00640 1 -- 172.21.15.16:0/3906710020 shutdown_connections 2024-08-17T02:48:54.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.902+0000 7f9008e00640 1 --2- 172.21.15.16:0/3906710020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9004155c50 0x7f90040af0f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.902+0000 7f9008e00640 1 -- 172.21.15.16:0/3906710020 >> 172.21.15.16:0/3906710020 conn(0x7f9004067750 msgr2=0x7f90040cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:54.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.902+0000 7f9008e00640 1 -- 172.21.15.16:0/3906710020 shutdown_connections 2024-08-17T02:48:54.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.902+0000 7f9008e00640 1 -- 172.21.15.16:0/3906710020 wait complete. 2024-08-17T02:48:54.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.904+0000 7f9008e00640 1 Processor -- start 2024-08-17T02:48:54.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.904+0000 7f9008e00640 1 -- start start 2024-08-17T02:48:54.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.905+0000 7f9008e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90040af630 0x7f90041541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.905+0000 7f9008e00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9004155880 0x7f90040c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.905+0000 7f9008e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9004155c50 0x7f900414d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.905+0000 7f9008e00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90040c6530 con 0x7f9004155c50 2024-08-17T02:48:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.905+0000 7f9008e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90040c6810 con 0x7f90040af630 2024-08-17T02:48:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.905+0000 7f9008e00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90040c6af0 con 0x7f9004155880 2024-08-17T02:48:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.905+0000 7f9003e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9004155c50 0x7f900414d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.905+0000 7f9003e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9004155c50 0x7f900414d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36636/0 (socket says 172.21.15.16:36636) 2024-08-17T02:48:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.905+0000 7f9003e00640 1 -- 172.21.15.16:0/777911810 learned_addr learned my addr 172.21.15.16:0/777911810 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f8ffbe00640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9004155880 0x7f90040c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f9003400640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90040af630 0x7f90041541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f9003e00640 1 -- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9004155880 msgr2=0x7f90040c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f9003e00640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f9004155880 0x7f90040c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f9003e00640 1 -- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90040af630 msgr2=0x7f90041541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f9003e00640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90040af630 0x7f90041541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f9003e00640 1 -- 172.21.15.16:0/777911810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec009390 con 0x7f9004155c50 2024-08-17T02:48:54.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f9003e00640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9004155c50 0x7f900414d180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00cc20 tx=0x7f8ffc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:54.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f8ffaa00640 1 -- 172.21.15.16:0/777911810 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00ce40 con 0x7f9004155c50 2024-08-17T02:48:54.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f8ffaa00640 1 -- 172.21.15.16:0/777911810 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc00a7f0 con 0x7f9004155c50 2024-08-17T02:48:54.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f8ffaa00640 1 -- 172.21.15.16:0/777911810 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00abb0 con 0x7f9004155c50 2024-08-17T02:48:54.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f9008e00640 1 -- 172.21.15.16:0/777911810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90040c6dd0 con 0x7f9004155c50 2024-08-17T02:48:54.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.906+0000 7f9008e00640 1 -- 172.21.15.16:0/777911810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90040c7720 con 0x7f9004155c50 2024-08-17T02:48:54.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.907+0000 7f9008e00640 1 -- 172.21.15.16:0/777911810 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9004071050 con 0x7f9004155c50 2024-08-17T02:48:54.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.908+0000 7f8ffaa00640 1 -- 172.21.15.16:0/777911810 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f8ffc0070f0 con 0x7f9004155c50 2024-08-17T02:48:54.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.909+0000 7f8ffaa00640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8fe0065150 0x7f8fe0067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.909+0000 7f8ffaa00640 1 -- 172.21.15.16:0/777911810 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f8ffc082350 con 0x7f9004155c50 2024-08-17T02:48:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.909+0000 7f8ffaa00640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] conn(0x7f8fe006b040 0x7f8fe006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.909+0000 7f8ffaa00640 1 -- 172.21.15.16:0/777911810 --> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fe006db00 con 0x7f8fe006b040 2024-08-17T02:48:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.909+0000 7f8ffaa00640 1 -- 172.21.15.16:0/777911810 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ffc0848b0 con 0x7f9004155c50 2024-08-17T02:48:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.910+0000 7f9003400640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8fe0065150 0x7f8fe0067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.910+0000 7f8ffbe00640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] conn(0x7f8fe006b040 0x7f8fe006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.911+0000 7f9003400640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8fe0065150 0x7f8fe0067600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40037e0 tx=0x7f8ff4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.911+0000 7f8ffbe00640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] conn(0x7f8fe006b040 0x7f8fe006d450 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.912+0000 7f8ffaa00640 1 -- 172.21.15.16:0/777911810 <== osd.4 v2:172.21.15.142:6818/37611 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f8fe006db00 con 0x7f8fe006b040 2024-08-17T02:48:54.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.925+0000 7f9008e00640 1 -- 172.21.15.16:0/777911810 --> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f90040af180 con 0x7f8fe006b040 2024-08-17T02:48:54.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.925+0000 7f8ffaa00640 1 -- 172.21.15.16:0/777911810 <== osd.4 v2:172.21.15.142:6818/37611 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f90040af180 con 0x7f8fe006b040 2024-08-17T02:48:54.927 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-17T02:48:54.927 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:48:54.927 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-17T02:48:54.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.926+0000 7f9008e00640 1 -- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] conn(0x7f8fe006b040 msgr2=0x7f8fe006d450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.926+0000 7f9008e00640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:6818/37611,v1:172.21.15.142:6820/37611] conn(0x7f8fe006b040 0x7f8fe006d450 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:54.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.926+0000 7f9008e00640 1 -- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8fe0065150 msgr2=0x7f8fe0067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.926+0000 7f9008e00640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f8fe0065150 0x7f8fe0067600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40037e0 tx=0x7f8ff4009040 comp rx=0 tx=0).stop 2024-08-17T02:48:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.926+0000 7f9008e00640 1 -- 172.21.15.16:0/777911810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9004155c50 msgr2=0x7f900414d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.926+0000 7f9008e00640 1 --2- 172.21.15.16:0/777911810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9004155c50 0x7f900414d180 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00cc20 tx=0x7f8ffc001a40 comp rx=0 tx=0).stop 2024-08-17T02:48:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.926+0000 7f9003e00640 1 -- 172.21.15.16:0/777911810 reap_dead start 2024-08-17T02:48:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.926+0000 7f9008e00640 1 -- 172.21.15.16:0/777911810 shutdown_connections 2024-08-17T02:48:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.926+0000 7f9008e00640 1 -- 172.21.15.16:0/777911810 >> 172.21.15.16:0/777911810 conn(0x7f9004067750 msgr2=0x7f90041587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.926+0000 7f9008e00640 1 -- 172.21.15.16:0/777911810 shutdown_connections 2024-08-17T02:48:54.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:54.926+0000 7f9008e00640 1 -- 172.21.15.16:0/777911810 wait complete. 2024-08-17T02:48:54.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-08-17T02:48:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.093+0000 7f5bd6600640 1 Processor -- start 2024-08-17T02:48:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.093+0000 7f5bd6600640 1 -- start start 2024-08-17T02:48:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.094+0000 7f5bd6600640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bd0159ca0 0x7f5bd00ca360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.094+0000 7f5bd6600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bd00ca8a0 0x7f5bd00cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.094+0000 7f5bd6600640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd00cadc0 con 0x7f5bd00ccad0 2024-08-17T02:48:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.094+0000 7f5bd6600640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd00cd6e0 con 0x7f5bd0159ca0 2024-08-17T02:48:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.094+0000 7f5bd6600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd00ccf70 con 0x7f5bd00ca8a0 2024-08-17T02:48:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.094+0000 7f5bd5200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bd0159ca0 0x7f5bd00ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.094+0000 7f5bd5c00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5bd00ccad0 0x7f5bd00cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34802/0 (socket says 172.21.15.16:34802) 2024-08-17T02:48:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.094+0000 7f5bd5c00640 1 -- 172.21.15.16:0/2880968977 learned_addr learned my addr 172.21.15.16:0/2880968977 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.094+0000 7f5bcfe00640 1 --2- 172.21.15.16:0/2880968977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bd00ca8a0 0x7f5bd00cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.095+0000 7f5bd5200640 1 -- 172.21.15.16:0/2880968977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bd00ca8a0 msgr2=0x7f5bd00cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.095+0000 7f5bd5200640 1 --2- 172.21.15.16:0/2880968977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bd00ca8a0 0x7f5bd00cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.095+0000 7f5bd5200640 1 -- 172.21.15.16:0/2880968977 >> v1:172.21.15.16:6789/0 conn(0x7f5bd00ccad0 legacy=0x7f5bd00cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.095+0000 7f5bd5200640 1 -- 172.21.15.16:0/2880968977 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd00cb060 con 0x7f5bd0159ca0 2024-08-17T02:48:55.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.095+0000 7f5bcf400640 1 -- 172.21.15.16:0/2880968977 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152351174 0 0) 0x7f5bd00cadc0 con 0x7f5bd00ccad0 2024-08-17T02:48:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.096+0000 7f5bd5200640 1 --2- 172.21.15.16:0/2880968977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd0159ca0 0x7f5bd00ca360 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009850 tx=0x7f5bc40090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a917065ba8f1ccd server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:55.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.096+0000 7f5bcf400640 1 -- 172.21.15.16:0/2880968977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc402ad90 con 0x7f5bd0159ca0 2024-08-17T02:48:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.096+0000 7f5bcf400640 1 -- 172.21.15.16:0/2880968977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc4032440 con 0x7f5bd0159ca0 2024-08-17T02:48:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.096+0000 7f5bcf400640 1 -- 172.21.15.16:0/2880968977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40327e0 con 0x7f5bd0159ca0 2024-08-17T02:48:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.096+0000 7f5bd6600640 1 -- 172.21.15.16:0/2880968977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd0159ca0 msgr2=0x7f5bd00ca360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.096+0000 7f5bd6600640 1 --2- 172.21.15.16:0/2880968977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd0159ca0 0x7f5bd00ca360 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009850 tx=0x7f5bc40090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.096+0000 7f5bd6600640 1 -- 172.21.15.16:0/2880968977 shutdown_connections 2024-08-17T02:48:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.096+0000 7f5bd6600640 1 --2- 172.21.15.16:0/2880968977 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bd00ca8a0 0x7f5bd00cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.096+0000 7f5bd6600640 1 --2- 172.21.15.16:0/2880968977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd0159ca0 0x7f5bd00ca360 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.096+0000 7f5bd6600640 1 -- 172.21.15.16:0/2880968977 >> 172.21.15.16:0/2880968977 conn(0x7f5bd0098e00 msgr2=0x7f5bd006eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.097+0000 7f5bd6600640 1 -- 172.21.15.16:0/2880968977 shutdown_connections 2024-08-17T02:48:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.097+0000 7f5bd6600640 1 -- 172.21.15.16:0/2880968977 wait complete. 2024-08-17T02:48:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.098+0000 7f5bd6600640 1 Processor -- start 2024-08-17T02:48:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.098+0000 7f5bd6600640 1 -- start start 2024-08-17T02:48:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.099+0000 7f5bd6600640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bd00ca8a0 0x7f5bd00c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.099+0000 7f5bd6600640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd00ccad0 0x7f5bd00c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.099+0000 7f5bd6600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bd0159ca0 0x7f5bd016bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.099+0000 7f5bd6600640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd00c66d0 con 0x7f5bd00ccad0 2024-08-17T02:48:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.099+0000 7f5bd6600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd00c8150 con 0x7f5bd0159ca0 2024-08-17T02:48:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.099+0000 7f5bd6600640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd00c8430 con 0x7f5bd00ca8a0 2024-08-17T02:48:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.099+0000 7f5bcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd00ccad0 0x7f5bd00c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.099+0000 7f5bcfe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd00ccad0 0x7f5bd00c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36658/0 (socket says 172.21.15.16:36658) 2024-08-17T02:48:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.099+0000 7f5bcfe00640 1 -- 172.21.15.16:0/1644758680 learned_addr learned my addr 172.21.15.16:0/1644758680 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.099+0000 7f5bd5200640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bd00ca8a0 0x7f5bd00c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.099+0000 7f5bd5c00640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bd0159ca0 0x7f5bd016bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.100+0000 7f5bcfe00640 1 -- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bd00ca8a0 msgr2=0x7f5bd00c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.100+0000 7f5bcfe00640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bd00ca8a0 0x7f5bd00c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.100+0000 7f5bcfe00640 1 -- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bd0159ca0 msgr2=0x7f5bd016bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.100+0000 7f5bcfe00640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bd0159ca0 0x7f5bd016bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.100+0000 7f5bcfe00640 1 -- 172.21.15.16:0/1644758680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc4009390 con 0x7f5bd00ccad0 2024-08-17T02:48:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.100+0000 7f5bcfe00640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd00ccad0 0x7f5bd00c6190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00ea20 tx=0x7f5bbc0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.100+0000 7f5bc3e00640 1 -- 172.21.15.16:0/1644758680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc00ca10 con 0x7f5bd00ccad0 2024-08-17T02:48:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.100+0000 7f5bc3e00640 1 -- 172.21.15.16:0/1644758680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bbc00cb70 con 0x7f5bd00ccad0 2024-08-17T02:48:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.100+0000 7f5bc3e00640 1 -- 172.21.15.16:0/1644758680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc013470 con 0x7f5bd00ccad0 2024-08-17T02:48:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.100+0000 7f5bd6600640 1 -- 172.21.15.16:0/1644758680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd00c8710 con 0x7f5bd00ccad0 2024-08-17T02:48:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.100+0000 7f5bd6600640 1 -- 172.21.15.16:0/1644758680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd00c8cd0 con 0x7f5bd00ccad0 2024-08-17T02:48:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.101+0000 7f5bd6600640 1 -- 172.21.15.16:0/1644758680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd0071050 con 0x7f5bd00ccad0 2024-08-17T02:48:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.102+0000 7f5bc3e00640 1 -- 172.21.15.16:0/1644758680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f5bbc013860 con 0x7f5bd00ccad0 2024-08-17T02:48:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.103+0000 7f5bc3e00640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5bb00651d0 0x7f5bb0067680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.103+0000 7f5bc3e00640 1 -- 172.21.15.16:0/1644758680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f5bbc0825f0 con 0x7f5bd00ccad0 2024-08-17T02:48:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.103+0000 7f5bd5200640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5bb00651d0 0x7f5bb0067680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.104+0000 7f5bd5200640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5bb00651d0 0x7f5bb0067680 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4034000 tx=0x7f5bc4032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.107+0000 7f5bc3e00640 1 -- 172.21.15.16:0/1644758680 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f5bbc050a20 con 0x7f5bd00ccad0 2024-08-17T02:48:55.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.287+0000 7f5bd6600640 1 -- 172.21.15.16:0/1644758680 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bd006ef40 con 0x7f5bb00651d0 2024-08-17T02:48:55.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.288+0000 7f5bc3e00640 1 -- 172.21.15.16:0/1644758680 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5bd006ef40 con 0x7f5bb00651d0 2024-08-17T02:48:55.289 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 4 to deep-scrub 2024-08-17T02:48:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.290+0000 7f5bd6600640 1 -- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5bb00651d0 msgr2=0x7f5bb0067680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.290+0000 7f5bd6600640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5bb00651d0 0x7f5bb0067680 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4034000 tx=0x7f5bc4032000 comp rx=0 tx=0).stop 2024-08-17T02:48:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.290+0000 7f5bd6600640 1 -- 172.21.15.16:0/1644758680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd00ccad0 msgr2=0x7f5bd00c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.290+0000 7f5bd6600640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd00ccad0 0x7f5bd00c6190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00ea20 tx=0x7f5bbc0090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.290+0000 7f5bd6600640 1 -- 172.21.15.16:0/1644758680 shutdown_connections 2024-08-17T02:48:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.290+0000 7f5bd6600640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f5bb00651d0 0x7f5bb0067680 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.290+0000 7f5bd6600640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bd0159ca0 0x7f5bd016bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.290+0000 7f5bd6600640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd00ccad0 0x7f5bd00c6190 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.290+0000 7f5bd6600640 1 --2- 172.21.15.16:0/1644758680 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f5bd00ca8a0 0x7f5bd00c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.290+0000 7f5bd6600640 1 -- 172.21.15.16:0/1644758680 >> 172.21.15.16:0/1644758680 conn(0x7f5bd0098e00 msgr2=0x7f5bd0157520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:55.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.290+0000 7f5bd6600640 1 -- 172.21.15.16:0/1644758680 shutdown_connections 2024-08-17T02:48:55.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.291+0000 7f5bd6600640 1 -- 172.21.15.16:0/1644758680 wait complete. 2024-08-17T02:48:55.306 INFO:tasks.ceph:Scrubbing osd.5 2024-08-17T02:48:55.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:48:55.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.461+0000 7fa31ee00640 1 Processor -- start 2024-08-17T02:48:55.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.461+0000 7fa31ee00640 1 -- start start 2024-08-17T02:48:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.462+0000 7fa31ee00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa318159180 0x7fa3180caa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.462+0000 7fa31ee00640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3180cb4c0 con 0x7fa318158db0 2024-08-17T02:48:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.462+0000 7fa31ee00640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3180cf770 con 0x7fa3180cf290 2024-08-17T02:48:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.462+0000 7fa31ee00640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3180cfa50 con 0x7fa318159180 2024-08-17T02:48:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.462+0000 7fa317e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa318158db0 0x7fa3180ca330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34818/0 (socket says 172.21.15.16:34818) 2024-08-17T02:48:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.462+0000 7fa317e00640 1 -- 172.21.15.16:0/776015 learned_addr learned my addr 172.21.15.16:0/776015 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.462+0000 7fa31ce00640 1 --2- 172.21.15.16:0/776015 >> v2:172.21.15.16:3300/0 conn(0x7fa318159180 0x7fa3180caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.463+0000 7fa31ce00640 1 -- 172.21.15.16:0/776015 >> v1:172.21.15.142:6789/0 conn(0x7fa3180cf290 legacy=0x7fa3180c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.463+0000 7fa31ce00640 1 -- 172.21.15.16:0/776015 >> v1:172.21.15.16:6789/0 conn(0x7fa318158db0 legacy=0x7fa3180ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.463+0000 7fa31ce00640 1 -- 172.21.15.16:0/776015 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3180cb150 con 0x7fa318159180 2024-08-17T02:48:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.463+0000 7fa317400640 1 -- 172.21.15.16:0/776015 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057425791 0 0) 0x7fa3180cb4c0 con 0x7fa318158db0 2024-08-17T02:48:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.464+0000 7fa31ce00640 1 --2- 172.21.15.16:0/776015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa318159180 0x7fa3180caa40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa310009e90 tx=0x7fa310009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f52b3679fd2071 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.464+0000 7fa317400640 1 -- 172.21.15.16:0/776015 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310007550 con 0x7fa318159180 2024-08-17T02:48:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.464+0000 7fa317400640 1 -- 172.21.15.16:0/776015 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3100076b0 con 0x7fa318159180 2024-08-17T02:48:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.464+0000 7fa317400640 1 -- 172.21.15.16:0/776015 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310007a50 con 0x7fa318159180 2024-08-17T02:48:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.465+0000 7fa31ee00640 1 -- 172.21.15.16:0/776015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa318159180 msgr2=0x7fa3180caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.465+0000 7fa31ee00640 1 --2- 172.21.15.16:0/776015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa318159180 0x7fa3180caa40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa310009e90 tx=0x7fa310009780 comp rx=0 tx=0).stop 2024-08-17T02:48:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.465+0000 7fa31ee00640 1 -- 172.21.15.16:0/776015 shutdown_connections 2024-08-17T02:48:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.465+0000 7fa31ee00640 1 --2- 172.21.15.16:0/776015 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa318159180 0x7fa3180caa40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.465+0000 7fa31ee00640 1 -- 172.21.15.16:0/776015 >> 172.21.15.16:0/776015 conn(0x7fa3180cfd90 msgr2=0x7fa3180d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.465+0000 7fa31ee00640 1 -- 172.21.15.16:0/776015 shutdown_connections 2024-08-17T02:48:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.465+0000 7fa31ee00640 1 -- 172.21.15.16:0/776015 wait complete. 2024-08-17T02:48:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.466+0000 7fa31ee00640 1 Processor -- start 2024-08-17T02:48:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.466+0000 7fa31ee00640 1 -- start start 2024-08-17T02:48:55.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.467+0000 7fa31ee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3180cf290 0x7fa3180c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.467+0000 7fa31ee00640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa318158db0 0x7fa31816ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.467+0000 7fa31ee00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa318159180 0x7fa31816cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.467+0000 7fa31ee00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3180c6750 con 0x7fa318159180 2024-08-17T02:48:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.467+0000 7fa31ee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3180b57c0 con 0x7fa3180cf290 2024-08-17T02:48:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.467+0000 7fa31ee00640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3180b4e90 con 0x7fa318158db0 2024-08-17T02:48:55.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.468+0000 7fa31ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa318159180 0x7fa31816cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.468+0000 7fa31ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa318159180 0x7fa31816cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36680/0 (socket says 172.21.15.16:36680) 2024-08-17T02:48:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.468+0000 7fa31ce00640 1 -- 172.21.15.16:0/3238512631 learned_addr learned my addr 172.21.15.16:0/3238512631 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.468+0000 7fa317e00640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3180cf290 0x7fa3180c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.468+0000 7fa30fe00640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa318158db0 0x7fa31816ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.468+0000 7fa31ce00640 1 -- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa318158db0 msgr2=0x7fa31816ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.468+0000 7fa31ce00640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa318158db0 0x7fa31816ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.468+0000 7fa31ce00640 1 -- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3180cf290 msgr2=0x7fa3180c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.468+0000 7fa31ce00640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3180cf290 0x7fa3180c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.468+0000 7fa31ce00640 1 -- 172.21.15.16:0/3238512631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa310009a20 con 0x7fa318159180 2024-08-17T02:48:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.468+0000 7fa31ce00640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa318159180 0x7fa31816cfd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa310009e90 tx=0x7fa310038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.469+0000 7fa30f400640 1 -- 172.21.15.16:0/3238512631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310032e70 con 0x7fa318159180 2024-08-17T02:48:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.469+0000 7fa30f400640 1 -- 172.21.15.16:0/3238512631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31003d420 con 0x7fa318159180 2024-08-17T02:48:55.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.469+0000 7fa30f400640 1 -- 172.21.15.16:0/3238512631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31003d7c0 con 0x7fa318159180 2024-08-17T02:48:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.469+0000 7fa31ee00640 1 -- 172.21.15.16:0/3238512631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3180b5170 con 0x7fa318159180 2024-08-17T02:48:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.469+0000 7fa31ee00640 1 -- 172.21.15.16:0/3238512631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3180c63d0 con 0x7fa318159180 2024-08-17T02:48:55.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.470+0000 7fa31ee00640 1 -- 172.21.15.16:0/3238512631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa3180710c0 con 0x7fa318159180 2024-08-17T02:48:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.471+0000 7fa30f400640 1 -- 172.21.15.16:0/3238512631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fa31003dbf0 con 0x7fa318159180 2024-08-17T02:48:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.472+0000 7fa30f400640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa2f4065150 0x7fa2f4067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.472+0000 7fa30f400640 1 -- 172.21.15.16:0/3238512631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fa3100a3e70 con 0x7fa318159180 2024-08-17T02:48:55.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.472+0000 7fa30f400640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] conn(0x7fa2f406b040 0x7fa2f406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.472+0000 7fa30f400640 1 -- 172.21.15.16:0/3238512631 --> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2f406db00 con 0x7fa2f406b040 2024-08-17T02:48:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.472+0000 7fa30f400640 1 -- 172.21.15.16:0/3238512631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3100a63d0 con 0x7fa318159180 2024-08-17T02:48:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.472+0000 7fa317e00640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa2f4065150 0x7fa2f4067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.472+0000 7fa30fe00640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] conn(0x7fa2f406b040 0x7fa2f406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.473+0000 7fa317e00640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa2f4065150 0x7fa2f4067600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa3080011e0 tx=0x7fa308009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.473+0000 7fa30fe00640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] conn(0x7fa2f406b040 0x7fa2f406d450 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:55.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.474+0000 7fa30f400640 1 -- 172.21.15.16:0/3238512631 <== osd.5 v2:172.21.15.142:6800/37609 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa2f406db00 con 0x7fa2f406b040 2024-08-17T02:48:55.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.487+0000 7fa31ee00640 1 -- 172.21.15.16:0/3238512631 --> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa318069200 con 0x7fa2f406b040 2024-08-17T02:48:55.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.487+0000 7fa30f400640 1 -- 172.21.15.16:0/3238512631 <== osd.5 v2:172.21.15.142:6800/37609 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa318069200 con 0x7fa2f406b040 2024-08-17T02:48:55.489 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-17T02:48:55.489 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:48:55.489 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-17T02:48:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.488+0000 7fa31ee00640 1 -- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] conn(0x7fa2f406b040 msgr2=0x7fa2f406d450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.488+0000 7fa31ee00640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:6800/37609,v1:172.21.15.142:6801/37609] conn(0x7fa2f406b040 0x7fa2f406d450 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.488+0000 7fa31ee00640 1 -- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa2f4065150 msgr2=0x7fa2f4067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.488+0000 7fa31ee00640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa2f4065150 0x7fa2f4067600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa3080011e0 tx=0x7fa308009040 comp rx=0 tx=0).stop 2024-08-17T02:48:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.488+0000 7fa31ee00640 1 -- 172.21.15.16:0/3238512631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa318159180 msgr2=0x7fa31816cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.488+0000 7fa31ee00640 1 --2- 172.21.15.16:0/3238512631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa318159180 0x7fa31816cfd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa310009e90 tx=0x7fa310038000 comp rx=0 tx=0).stop 2024-08-17T02:48:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.488+0000 7fa31ce00640 1 -- 172.21.15.16:0/3238512631 reap_dead start 2024-08-17T02:48:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.488+0000 7fa31ee00640 1 -- 172.21.15.16:0/3238512631 shutdown_connections 2024-08-17T02:48:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.488+0000 7fa31ee00640 1 -- 172.21.15.16:0/3238512631 >> 172.21.15.16:0/3238512631 conn(0x7fa3180cfd90 msgr2=0x7fa3180c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:55.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.488+0000 7fa31ee00640 1 -- 172.21.15.16:0/3238512631 shutdown_connections 2024-08-17T02:48:55.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.488+0000 7fa31ee00640 1 -- 172.21.15.16:0/3238512631 wait complete. 2024-08-17T02:48:55.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-08-17T02:48:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.654+0000 7fde7f800640 1 Processor -- start 2024-08-17T02:48:55.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.654+0000 7fde7f800640 1 -- start start 2024-08-17T02:48:55.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.655+0000 7fde7f800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde7815a260 0x7fde780b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.655+0000 7fde7f800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fde78096a70 0x7fde780ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.655+0000 7fde7f800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde780aba20 0x7fde780b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.655+0000 7fde7f800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde780abea0 con 0x7fde780aba20 2024-08-17T02:48:55.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.655+0000 7fde7f800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde780adb60 con 0x7fde78096a70 2024-08-17T02:48:55.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.655+0000 7fde7f800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde780975a0 con 0x7fde7815a260 2024-08-17T02:48:55.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.655+0000 7fde7d800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde780aba20 0x7fde780b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.656+0000 7fde7d800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde780aba20 0x7fde780b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36690/0 (socket says 172.21.15.16:36690) 2024-08-17T02:48:55.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.656+0000 7fde7d800640 1 -- 172.21.15.16:0/2116938214 learned_addr learned my addr 172.21.15.16:0/2116938214 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:55.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.656+0000 7fde77e00640 1 --2- 172.21.15.16:0/2116938214 >> v2:172.21.15.142:3300/0 conn(0x7fde78096a70 0x7fde780ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.656+0000 7fde7ce00640 1 --2- 172.21.15.16:0/2116938214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde7815a260 0x7fde780b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.656+0000 7fde7d800640 1 -- 172.21.15.16:0/2116938214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde7815a260 msgr2=0x7fde780b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:55.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.656+0000 7fde7d800640 1 --2- 172.21.15.16:0/2116938214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde7815a260 0x7fde780b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.656+0000 7fde7d800640 1 -- 172.21.15.16:0/2116938214 >> v2:172.21.15.142:3300/0 conn(0x7fde78096a70 msgr2=0x7fde780ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:55.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.656+0000 7fde7d800640 1 --2- 172.21.15.16:0/2116938214 >> v2:172.21.15.142:3300/0 conn(0x7fde78096a70 0x7fde780ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.656+0000 7fde7d800640 1 -- 172.21.15.16:0/2116938214 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7815a630 con 0x7fde780aba20 2024-08-17T02:48:55.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.657+0000 7fde7d800640 1 --2- 172.21.15.16:0/2116938214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde780aba20 0x7fde780b0980 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fde78067db0 tx=0x7fde70009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f645932aca7a611 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:55.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.657+0000 7fde77400640 1 -- 172.21.15.16:0/2116938214 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70031400 con 0x7fde780aba20 2024-08-17T02:48:55.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.657+0000 7fde77400640 1 -- 172.21.15.16:0/2116938214 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7002d050 con 0x7fde780aba20 2024-08-17T02:48:55.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.657+0000 7fde77400640 1 -- 172.21.15.16:0/2116938214 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde700317d0 con 0x7fde780aba20 2024-08-17T02:48:55.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.658+0000 7fde7f800640 1 -- 172.21.15.16:0/2116938214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde780aba20 msgr2=0x7fde780b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.658+0000 7fde7f800640 1 --2- 172.21.15.16:0/2116938214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde780aba20 0x7fde780b0980 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fde78067db0 tx=0x7fde70009840 comp rx=0 tx=0).stop 2024-08-17T02:48:55.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.658+0000 7fde7f800640 1 -- 172.21.15.16:0/2116938214 shutdown_connections 2024-08-17T02:48:55.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.658+0000 7fde7f800640 1 --2- 172.21.15.16:0/2116938214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde780aba20 0x7fde780b0980 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.658+0000 7fde7f800640 1 --2- 172.21.15.16:0/2116938214 >> v2:172.21.15.142:3300/0 conn(0x7fde78096a70 0x7fde780ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.658+0000 7fde7f800640 1 --2- 172.21.15.16:0/2116938214 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde7815a260 0x7fde780b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.658+0000 7fde7f800640 1 -- 172.21.15.16:0/2116938214 >> 172.21.15.16:0/2116938214 conn(0x7fde780b3040 msgr2=0x7fde780b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:55.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.658+0000 7fde7f800640 1 -- 172.21.15.16:0/2116938214 shutdown_connections 2024-08-17T02:48:55.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.658+0000 7fde7f800640 1 -- 172.21.15.16:0/2116938214 wait complete. 2024-08-17T02:48:55.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.659+0000 7fde7f800640 1 Processor -- start 2024-08-17T02:48:55.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.660+0000 7fde7f800640 1 -- start start 2024-08-17T02:48:55.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.660+0000 7fde7f800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde78096a70 0x7fde78146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.660+0000 7fde7f800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde780aba20 0x7fde7814da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.660+0000 7fde7f800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7815a260 0x7fde78098440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde7f800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde78099560 con 0x7fde7815a260 2024-08-17T02:48:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde7f800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7809a300 con 0x7fde780aba20 2024-08-17T02:48:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde7f800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7814f440 con 0x7fde78096a70 2024-08-17T02:48:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde7d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7815a260 0x7fde78098440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde7d800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7815a260 0x7fde78098440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36698/0 (socket says 172.21.15.16:36698) 2024-08-17T02:48:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde7d800640 1 -- 172.21.15.16:0/3651570418 learned_addr learned my addr 172.21.15.16:0/3651570418 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde7ce00640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde78096a70 0x7fde78146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde77e00640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde780aba20 0x7fde7814da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde7d800640 1 -- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde78096a70 msgr2=0x7fde78146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde7d800640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde78096a70 0x7fde78146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde7d800640 1 -- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde780aba20 msgr2=0x7fde7814da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde7d800640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde780aba20 0x7fde7814da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.661+0000 7fde7d800640 1 -- 172.21.15.16:0/3651570418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde70009ae0 con 0x7fde7815a260 2024-08-17T02:48:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.662+0000 7fde7d800640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7815a260 0x7fde78098440 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fde70009f20 tx=0x7fde70036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.662+0000 7fde6be00640 1 -- 172.21.15.16:0/3651570418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70031880 con 0x7fde7815a260 2024-08-17T02:48:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.662+0000 7fde6be00640 1 -- 172.21.15.16:0/3651570418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde70007170 con 0x7fde7815a260 2024-08-17T02:48:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.662+0000 7fde7f800640 1 -- 172.21.15.16:0/3651570418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde780997b0 con 0x7fde7815a260 2024-08-17T02:48:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.662+0000 7fde6be00640 1 -- 172.21.15.16:0/3651570418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7003b640 con 0x7fde7815a260 2024-08-17T02:48:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.662+0000 7fde7f800640 1 -- 172.21.15.16:0/3651570418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde78099f90 con 0x7fde7815a260 2024-08-17T02:48:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.663+0000 7fde7f800640 1 -- 172.21.15.16:0/3651570418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7809a4b0 con 0x7fde7815a260 2024-08-17T02:48:55.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.664+0000 7fde6be00640 1 -- 172.21.15.16:0/3651570418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fde70038050 con 0x7fde7815a260 2024-08-17T02:48:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.665+0000 7fde6be00640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fde58065180 0x7fde58067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.665+0000 7fde6be00640 1 -- 172.21.15.16:0/3651570418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fde700a2530 con 0x7fde7815a260 2024-08-17T02:48:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.665+0000 7fde7ce00640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fde58065180 0x7fde58067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.666+0000 7fde7ce00640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fde58065180 0x7fde58067630 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009c10 tx=0x7fde6c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.669+0000 7fde6be00640 1 -- 172.21.15.16:0/3651570418 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fde7809a4b0 con 0x7fde7815a260 2024-08-17T02:48:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.855+0000 7fde7f800640 1 -- 172.21.15.16:0/3651570418 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fde78099de0 con 0x7fde58065180 2024-08-17T02:48:55.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.855+0000 7fde6be00640 1 -- 172.21.15.16:0/3651570418 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fde78099de0 con 0x7fde58065180 2024-08-17T02:48:55.857 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 5 to deep-scrub 2024-08-17T02:48:55.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.857+0000 7fde7f800640 1 -- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fde58065180 msgr2=0x7fde58067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.857+0000 7fde7f800640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fde58065180 0x7fde58067630 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009c10 tx=0x7fde6c008040 comp rx=0 tx=0).stop 2024-08-17T02:48:55.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.857+0000 7fde7f800640 1 -- 172.21.15.16:0/3651570418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7815a260 msgr2=0x7fde78098440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:55.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.857+0000 7fde7f800640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7815a260 0x7fde78098440 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fde70009f20 tx=0x7fde70036000 comp rx=0 tx=0).stop 2024-08-17T02:48:55.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.857+0000 7fde7f800640 1 -- 172.21.15.16:0/3651570418 shutdown_connections 2024-08-17T02:48:55.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.857+0000 7fde7f800640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fde58065180 0x7fde58067630 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.857+0000 7fde7f800640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde7815a260 0x7fde78098440 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.857+0000 7fde7f800640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde780aba20 0x7fde7814da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.857+0000 7fde7f800640 1 --2- 172.21.15.16:0/3651570418 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde78096a70 0x7fde78146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.857+0000 7fde7f800640 1 -- 172.21.15.16:0/3651570418 >> 172.21.15.16:0/3651570418 conn(0x7fde780b3040 msgr2=0x7fde78158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.857+0000 7fde7f800640 1 -- 172.21.15.16:0/3651570418 shutdown_connections 2024-08-17T02:48:55.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:55.858+0000 7fde7f800640 1 -- 172.21.15.16:0/3651570418 wait complete. 2024-08-17T02:48:55.873 INFO:tasks.ceph:Scrubbing osd.6 2024-08-17T02:48:55.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:48:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.028+0000 7fa065200640 1 Processor -- start 2024-08-17T02:48:56.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.028+0000 7fa065200640 1 -- start start 2024-08-17T02:48:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.029+0000 7fa065200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7fa060150130 0x7fa0600aa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.029+0000 7fa065200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0601568b0 con 0x7fa06006e5f0 2024-08-17T02:48:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.029+0000 7fa065200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa060150500 con 0x7fa060156d40 2024-08-17T02:48:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.029+0000 7fa065200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa060155fc0 con 0x7fa060150130 2024-08-17T02:48:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.030+0000 7fa05e800640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa06006e5f0 0x7fa0600ab510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34828/0 (socket says 172.21.15.16:34828) 2024-08-17T02:48:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.030+0000 7fa05e800640 1 -- 172.21.15.16:0/3327163995 learned_addr learned my addr 172.21.15.16:0/3327163995 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:56.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.030+0000 7fa05de00640 1 --2- 172.21.15.16:0/3327163995 >> v2:172.21.15.142:3300/0 conn(0x7fa060150130 0x7fa0600aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.030+0000 7fa05d400640 1 -- 172.21.15.16:0/3327163995 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189415189 0 0) 0x7fa0601568b0 con 0x7fa06006e5f0 2024-08-17T02:48:56.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.030+0000 7fa05d400640 1 -- 172.21.15.16:0/3327163995 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0440034e0 con 0x7fa06006e5f0 2024-08-17T02:48:56.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.030+0000 7fa05de00640 1 -- 172.21.15.16:0/3327163995 >> v1:172.21.15.142:6789/0 conn(0x7fa060156d40 legacy=0x7fa06006dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.030+0000 7fa05de00640 1 -- 172.21.15.16:0/3327163995 >> v1:172.21.15.16:6789/0 conn(0x7fa06006e5f0 legacy=0x7fa0600ab510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.030+0000 7fa05de00640 1 -- 172.21.15.16:0/3327163995 --> v2:172.21.15.142:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0600add60 con 0x7fa060150130 2024-08-17T02:48:56.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.031+0000 7fa05de00640 1 --2- 172.21.15.16:0/3327163995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa060150130 0x7fa0600aa730 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa050009850 tx=0x7fa0500090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c248f91600919a23 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.032+0000 7fa05d400640 1 -- 172.21.15.16:0/3327163995 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05002ad90 con 0x7fa060150130 2024-08-17T02:48:56.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.032+0000 7fa05d400640 1 -- 172.21.15.16:0/3327163995 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa050032440 con 0x7fa060150130 2024-08-17T02:48:56.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.032+0000 7fa05d400640 1 -- 172.21.15.16:0/3327163995 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0500327c0 con 0x7fa060150130 2024-08-17T02:48:56.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.032+0000 7fa065200640 1 -- 172.21.15.16:0/3327163995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa060150130 msgr2=0x7fa0600aa730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.033+0000 7fa065200640 1 --2- 172.21.15.16:0/3327163995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa060150130 0x7fa0600aa730 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa050009850 tx=0x7fa0500090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.033+0000 7fa065200640 1 -- 172.21.15.16:0/3327163995 shutdown_connections 2024-08-17T02:48:56.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.033+0000 7fa065200640 1 --2- 172.21.15.16:0/3327163995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa060150130 0x7fa0600aa730 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.033+0000 7fa065200640 1 -- 172.21.15.16:0/3327163995 >> 172.21.15.16:0/3327163995 conn(0x7fa060067750 msgr2=0x7fa06006c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:56.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.033+0000 7fa065200640 1 -- 172.21.15.16:0/3327163995 shutdown_connections 2024-08-17T02:48:56.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.033+0000 7fa065200640 1 -- 172.21.15.16:0/3327163995 wait complete. 2024-08-17T02:48:56.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.034+0000 7fa065200640 1 Processor -- start 2024-08-17T02:48:56.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.034+0000 7fa065200640 1 -- start start 2024-08-17T02:48:56.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.035+0000 7fa065200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa06006e5f0 0x7fa06014c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.035+0000 7fa065200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa060150130 0x7fa060158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.035+0000 7fa065200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa060156d40 0x7fa06015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.035+0000 7fa065200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06015cd50 con 0x7fa060156d40 2024-08-17T02:48:56.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.035+0000 7fa065200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600caaa0 con 0x7fa06006e5f0 2024-08-17T02:48:56.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.035+0000 7fa065200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600cad80 con 0x7fa060150130 2024-08-17T02:48:56.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.035+0000 7fa05e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa060156d40 0x7fa06015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.035+0000 7fa05e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa060156d40 0x7fa06015a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36704/0 (socket says 172.21.15.16:36704) 2024-08-17T02:48:56.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.035+0000 7fa05e800640 1 -- 172.21.15.16:0/366946882 learned_addr learned my addr 172.21.15.16:0/366946882 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:56.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.036+0000 7fa05de00640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa06006e5f0 0x7fa06014c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.036+0000 7fa055400640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa060150130 0x7fa060158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.036+0000 7fa05e800640 1 -- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa060150130 msgr2=0x7fa060158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.036+0000 7fa05e800640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa060150130 0x7fa060158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.036+0000 7fa05e800640 1 -- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa06006e5f0 msgr2=0x7fa06014c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.036+0000 7fa05e800640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa06006e5f0 0x7fa06014c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.036+0000 7fa05e800640 1 -- 172.21.15.16:0/366946882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa050009390 con 0x7fa060156d40 2024-08-17T02:48:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.036+0000 7fa055400640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fa060150130 0x7fa060158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.036+0000 7fa05e800640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa060156d40 0x7fa06015a310 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa05800cc40 tx=0x7fa058001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.036+0000 7fa056000640 1 -- 172.21.15.16:0/366946882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05800ce60 con 0x7fa060156d40 2024-08-17T02:48:56.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.036+0000 7fa056000640 1 -- 172.21.15.16:0/366946882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05800a850 con 0x7fa060156d40 2024-08-17T02:48:56.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.036+0000 7fa065200640 1 -- 172.21.15.16:0/366946882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0600cb060 con 0x7fa060156d40 2024-08-17T02:48:56.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.037+0000 7fa056000640 1 -- 172.21.15.16:0/366946882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05800ac10 con 0x7fa060156d40 2024-08-17T02:48:56.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.037+0000 7fa065200640 1 -- 172.21.15.16:0/366946882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa060158870 con 0x7fa060156d40 2024-08-17T02:48:56.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.037+0000 7fa065200640 1 -- 172.21.15.16:0/366946882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0600710c0 con 0x7fa060156d40 2024-08-17T02:48:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.038+0000 7fa056000640 1 -- 172.21.15.16:0/366946882 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fa058007300 con 0x7fa060156d40 2024-08-17T02:48:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.039+0000 7fa056000640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa03c065150 0x7fa03c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.039+0000 7fa056000640 1 -- 172.21.15.16:0/366946882 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fa058082280 con 0x7fa060156d40 2024-08-17T02:48:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.039+0000 7fa056000640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] conn(0x7fa03c06af70 0x7fa03c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.039+0000 7fa056000640 1 -- 172.21.15.16:0/366946882 --> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa03c06da30 con 0x7fa03c06af70 2024-08-17T02:48:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.039+0000 7fa056000640 1 -- 172.21.15.16:0/366946882 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0580848c0 con 0x7fa060156d40 2024-08-17T02:48:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.040+0000 7fa05de00640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa03c065150 0x7fa03c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.040+0000 7fa055400640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] conn(0x7fa03c06af70 0x7fa03c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.041+0000 7fa055400640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] conn(0x7fa03c06af70 0x7fa03c06d380 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.041+0000 7fa05de00640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa03c065150 0x7fa03c067600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa050034000 tx=0x7fa050032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.042+0000 7fa056000640 1 -- 172.21.15.16:0/366946882 <== osd.6 v2:172.21.15.142:6808/37610 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa03c06da30 con 0x7fa03c06af70 2024-08-17T02:48:56.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.054+0000 7fa065200640 1 -- 172.21.15.16:0/366946882 --> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa0600c8070 con 0x7fa03c06af70 2024-08-17T02:48:56.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.055+0000 7fa056000640 1 -- 172.21.15.16:0/366946882 <== osd.6 v2:172.21.15.142:6808/37610 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa0600c8070 con 0x7fa03c06af70 2024-08-17T02:48:56.056 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-17T02:48:56.057 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:48:56.057 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-17T02:48:56.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.055+0000 7fa065200640 1 -- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] conn(0x7fa03c06af70 msgr2=0x7fa03c06d380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.055+0000 7fa065200640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:6808/37610,v1:172.21.15.142:6809/37610] conn(0x7fa03c06af70 0x7fa03c06d380 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.055+0000 7fa065200640 1 -- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa03c065150 msgr2=0x7fa03c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.055+0000 7fa065200640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fa03c065150 0x7fa03c067600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa050034000 tx=0x7fa050032000 comp rx=0 tx=0).stop 2024-08-17T02:48:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.055+0000 7fa065200640 1 -- 172.21.15.16:0/366946882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa060156d40 msgr2=0x7fa06015a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.055+0000 7fa065200640 1 --2- 172.21.15.16:0/366946882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa060156d40 0x7fa06015a310 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa05800cc40 tx=0x7fa058001aa0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.056+0000 7fa05e800640 1 -- 172.21.15.16:0/366946882 reap_dead start 2024-08-17T02:48:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.056+0000 7fa065200640 1 -- 172.21.15.16:0/366946882 shutdown_connections 2024-08-17T02:48:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.056+0000 7fa065200640 1 -- 172.21.15.16:0/366946882 >> 172.21.15.16:0/366946882 conn(0x7fa060067750 msgr2=0x7fa06006dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.056+0000 7fa065200640 1 -- 172.21.15.16:0/366946882 shutdown_connections 2024-08-17T02:48:56.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.056+0000 7fa065200640 1 -- 172.21.15.16:0/366946882 wait complete. 2024-08-17T02:48:56.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-08-17T02:48:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.222+0000 7fe510800640 1 Processor -- start 2024-08-17T02:48:56.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.222+0000 7fe510800640 1 -- start start 2024-08-17T02:48:56.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.223+0000 7fe510800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe508155180 0x7fe5080aea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.223+0000 7fe510800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe50806e080 0x7fe50806e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.223+0000 7fe510800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5080afc10 con 0x7fe508155550 2024-08-17T02:48:56.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.223+0000 7fe510800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe50806ea00 con 0x7fe508155180 2024-08-17T02:48:56.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.223+0000 7fe510800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080af000 con 0x7fe50806e080 2024-08-17T02:48:56.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.224+0000 7fe50de00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe508155180 0x7fe5080aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.224+0000 7fe50de00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe508155180 0x7fe5080aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36720/0 (socket says 172.21.15.16:36720) 2024-08-17T02:48:56.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.224+0000 7fe50de00640 1 -- 172.21.15.16:0/3492894474 learned_addr learned my addr 172.21.15.16:0/3492894474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:56.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.224+0000 7fe50d400640 1 --1- 172.21.15.16:0/3492894474 >> v1:172.21.15.16:6789/0 conn(0x7fe508155550 0x7fe5080ae070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34840/0 (socket says 172.21.15.16:34840) 2024-08-17T02:48:56.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.224+0000 7fe50e800640 1 --2- 172.21.15.16:0/3492894474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe50806e080 0x7fe50806e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.224+0000 7fe50de00640 1 -- 172.21.15.16:0/3492894474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe50806e080 msgr2=0x7fe50806e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:56.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.224+0000 7fe50de00640 1 --2- 172.21.15.16:0/3492894474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe50806e080 0x7fe50806e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.224+0000 7fe50de00640 1 -- 172.21.15.16:0/3492894474 >> v1:172.21.15.16:6789/0 conn(0x7fe508155550 legacy=0x7fe5080ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.224+0000 7fe50de00640 1 -- 172.21.15.16:0/3492894474 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080af2e0 con 0x7fe508155180 2024-08-17T02:48:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.225+0000 7fe50de00640 1 --2- 172.21.15.16:0/3492894474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe508155180 0x7fe5080aea20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00bbc0 tx=0x7fe4fc00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2e0d5c2026a23c server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.226+0000 7fe50ca00640 1 -- 172.21.15.16:0/3492894474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc031610 con 0x7fe508155180 2024-08-17T02:48:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.226+0000 7fe50ca00640 1 -- 172.21.15.16:0/3492894474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4fc031770 con 0x7fe508155180 2024-08-17T02:48:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.226+0000 7fe50ca00640 1 -- 172.21.15.16:0/3492894474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc031b50 con 0x7fe508155180 2024-08-17T02:48:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.226+0000 7fe510800640 1 -- 172.21.15.16:0/3492894474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe508155180 msgr2=0x7fe5080aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.226+0000 7fe510800640 1 --2- 172.21.15.16:0/3492894474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe508155180 0x7fe5080aea20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00bbc0 tx=0x7fe4fc00b460 comp rx=0 tx=0).stop 2024-08-17T02:48:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.226+0000 7fe510800640 1 -- 172.21.15.16:0/3492894474 shutdown_connections 2024-08-17T02:48:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.226+0000 7fe510800640 1 --2- 172.21.15.16:0/3492894474 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe50806e080 0x7fe50806e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.226+0000 7fe510800640 1 --2- 172.21.15.16:0/3492894474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe508155180 0x7fe5080aea20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.226+0000 7fe510800640 1 -- 172.21.15.16:0/3492894474 >> 172.21.15.16:0/3492894474 conn(0x7fe5080cb9b0 msgr2=0x7fe5080cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.226+0000 7fe510800640 1 -- 172.21.15.16:0/3492894474 shutdown_connections 2024-08-17T02:48:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.226+0000 7fe510800640 1 -- 172.21.15.16:0/3492894474 wait complete. 2024-08-17T02:48:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.228+0000 7fe510800640 1 Processor -- start 2024-08-17T02:48:56.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.228+0000 7fe510800640 1 -- start start 2024-08-17T02:48:56.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.229+0000 7fe510800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe50806e080 0x7fe50814f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.229+0000 7fe510800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe508155180 0x7fe508158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.229+0000 7fe510800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe508155550 0x7fe508156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.229+0000 7fe510800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe508156660 con 0x7fe508155550 2024-08-17T02:48:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.229+0000 7fe510800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50814fcd0 con 0x7fe508155180 2024-08-17T02:48:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.229+0000 7fe510800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5081587d0 con 0x7fe50806e080 2024-08-17T02:48:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.229+0000 7fe50e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe508155550 0x7fe508156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.229+0000 7fe50e800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe508155550 0x7fe508156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36722/0 (socket says 172.21.15.16:36722) 2024-08-17T02:48:56.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.229+0000 7fe50e800640 1 -- 172.21.15.16:0/2905024064 learned_addr learned my addr 172.21.15.16:0/2905024064 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.229+0000 7fe50d400640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe508155180 0x7fe508158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.229+0000 7fe50de00640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe50806e080 0x7fe50814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.230+0000 7fe50e800640 1 -- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe50806e080 msgr2=0x7fe50814f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.230+0000 7fe50e800640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe50806e080 0x7fe50814f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.230+0000 7fe50e800640 1 -- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe508155180 msgr2=0x7fe508158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.230+0000 7fe50e800640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe508155180 0x7fe508158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.230+0000 7fe50e800640 1 -- 172.21.15.16:0/2905024064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc00b700 con 0x7fe508155550 2024-08-17T02:48:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.230+0000 7fe50e800640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe508155550 0x7fe508156af0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe504007f20 tx=0x7fe504009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.230+0000 7fe501600640 1 -- 172.21.15.16:0/2905024064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50400ee30 con 0x7fe508155550 2024-08-17T02:48:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.230+0000 7fe501600640 1 -- 172.21.15.16:0/2905024064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe504014420 con 0x7fe508155550 2024-08-17T02:48:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.230+0000 7fe501600640 1 -- 172.21.15.16:0/2905024064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5040147e0 con 0x7fe508155550 2024-08-17T02:48:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.230+0000 7fe510800640 1 -- 172.21.15.16:0/2905024064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe508155a50 con 0x7fe508155550 2024-08-17T02:48:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.230+0000 7fe510800640 1 -- 172.21.15.16:0/2905024064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe508156010 con 0x7fe508155550 2024-08-17T02:48:56.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.231+0000 7fe510800640 1 -- 172.21.15.16:0/2905024064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe508156810 con 0x7fe508155550 2024-08-17T02:48:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.232+0000 7fe501600640 1 -- 172.21.15.16:0/2905024064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fe504014b50 con 0x7fe508155550 2024-08-17T02:48:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.233+0000 7fe501600640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe4ec065180 0x7fe4ec067630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.233+0000 7fe501600640 1 -- 172.21.15.16:0/2905024064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fe5040827e0 con 0x7fe508155550 2024-08-17T02:48:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.233+0000 7fe50de00640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe4ec065180 0x7fe4ec067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.234+0000 7fe50de00640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe4ec065180 0x7fe4ec067630 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc037000 tx=0x7fe4fc03a000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.237+0000 7fe501600640 1 -- 172.21.15.16:0/2905024064 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fe508156810 con 0x7fe508155550 2024-08-17T02:48:56.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.418+0000 7fe510800640 1 -- 172.21.15.16:0/2905024064 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe508064db0 con 0x7fe4ec065180 2024-08-17T02:48:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.419+0000 7fe501600640 1 -- 172.21.15.16:0/2905024064 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe508064db0 con 0x7fe4ec065180 2024-08-17T02:48:56.420 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 6 to deep-scrub 2024-08-17T02:48:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.421+0000 7fe510800640 1 -- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe4ec065180 msgr2=0x7fe4ec067630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.422 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.421+0000 7fe510800640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe4ec065180 0x7fe4ec067630 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc037000 tx=0x7fe4fc03a000 comp rx=0 tx=0).stop 2024-08-17T02:48:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.421+0000 7fe510800640 1 -- 172.21.15.16:0/2905024064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe508155550 msgr2=0x7fe508156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.421+0000 7fe510800640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe508155550 0x7fe508156af0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe504007f20 tx=0x7fe504009860 comp rx=0 tx=0).stop 2024-08-17T02:48:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.421+0000 7fe510800640 1 -- 172.21.15.16:0/2905024064 shutdown_connections 2024-08-17T02:48:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.421+0000 7fe510800640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fe4ec065180 0x7fe4ec067630 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.421+0000 7fe510800640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe508155550 0x7fe508156af0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.421+0000 7fe510800640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe508155180 0x7fe508158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.421+0000 7fe510800640 1 --2- 172.21.15.16:0/2905024064 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fe50806e080 0x7fe50814f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.421+0000 7fe510800640 1 -- 172.21.15.16:0/2905024064 >> 172.21.15.16:0/2905024064 conn(0x7fe5080cb9b0 msgr2=0x7fe5081523c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.421+0000 7fe510800640 1 -- 172.21.15.16:0/2905024064 shutdown_connections 2024-08-17T02:48:56.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.421+0000 7fe510800640 1 -- 172.21.15.16:0/2905024064 wait complete. 2024-08-17T02:48:56.438 INFO:tasks.ceph:Scrubbing osd.7 2024-08-17T02:48:56.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-08-17T02:48:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.592+0000 7f1925000640 1 Processor -- start 2024-08-17T02:48:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.593+0000 7f1925000640 1 -- start start 2024-08-17T02:48:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.594+0000 7f1925000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19200b2ef0 0x7f19200b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.594+0000 7f1925000640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f19200b0830 0x7f19200b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.594+0000 7f1925000640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1920067de0 0x7f1920068220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.594+0000 7f1925000640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19200b1d20 con 0x7f1920067de0 2024-08-17T02:48:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.594+0000 7f1925000640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19200ad350 con 0x7f19200b0830 2024-08-17T02:48:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.594+0000 7f1925000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19200b3800 con 0x7f19200b2ef0 2024-08-17T02:48:56.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.594+0000 7f191fe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1920067de0 0x7f1920068220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.594+0000 7f191fe00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1920067de0 0x7f1920068220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36736/0 (socket says 172.21.15.16:36736) 2024-08-17T02:48:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.594+0000 7f191fe00640 1 -- 172.21.15.16:0/3222957633 learned_addr learned my addr 172.21.15.16:0/3222957633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.594+0000 7f191f400640 1 --2- 172.21.15.16:0/3222957633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19200b2ef0 0x7f19200b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.594+0000 7f1917e00640 1 --2- 172.21.15.16:0/3222957633 >> v2:172.21.15.142:3300/0 conn(0x7f19200b0830 0x7f19200b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.595+0000 7f191fe00640 1 -- 172.21.15.16:0/3222957633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19200b2ef0 msgr2=0x7f19200b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.595+0000 7f191fe00640 1 --2- 172.21.15.16:0/3222957633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19200b2ef0 0x7f19200b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.595+0000 7f191fe00640 1 -- 172.21.15.16:0/3222957633 >> v2:172.21.15.142:3300/0 conn(0x7f19200b0830 msgr2=0x7f19200b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:56.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.595+0000 7f191fe00640 1 --2- 172.21.15.16:0/3222957633 >> v2:172.21.15.142:3300/0 conn(0x7f19200b0830 0x7f19200b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.595+0000 7f191fe00640 1 -- 172.21.15.16:0/3222957633 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19200b0cf0 con 0x7f1920067de0 2024-08-17T02:48:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.596+0000 7f191fe00640 1 --2- 172.21.15.16:0/3222957633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1920067de0 0x7f1920068220 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f192006e1a0 tx=0x7f1918009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d2495b9854b282 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.596+0000 7f191ea00640 1 -- 172.21.15.16:0/3222957633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918031400 con 0x7f1920067de0 2024-08-17T02:48:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.596+0000 7f191ea00640 1 -- 172.21.15.16:0/3222957633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191802d050 con 0x7f1920067de0 2024-08-17T02:48:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.596+0000 7f191ea00640 1 -- 172.21.15.16:0/3222957633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19180317d0 con 0x7f1920067de0 2024-08-17T02:48:56.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.596+0000 7f1925000640 1 -- 172.21.15.16:0/3222957633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1920067de0 msgr2=0x7f1920068220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.596+0000 7f1925000640 1 --2- 172.21.15.16:0/3222957633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1920067de0 0x7f1920068220 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f192006e1a0 tx=0x7f1918009840 comp rx=0 tx=0).stop 2024-08-17T02:48:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.597+0000 7f1925000640 1 -- 172.21.15.16:0/3222957633 shutdown_connections 2024-08-17T02:48:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.597+0000 7f1925000640 1 --2- 172.21.15.16:0/3222957633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1920067de0 0x7f1920068220 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.597+0000 7f1925000640 1 --2- 172.21.15.16:0/3222957633 >> v2:172.21.15.142:3300/0 conn(0x7f19200b0830 0x7f19200b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.597+0000 7f1925000640 1 --2- 172.21.15.16:0/3222957633 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19200b2ef0 0x7f19200b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.597+0000 7f1925000640 1 -- 172.21.15.16:0/3222957633 >> 172.21.15.16:0/3222957633 conn(0x7f1920096a70 msgr2=0x7f1920096e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.597+0000 7f1925000640 1 -- 172.21.15.16:0/3222957633 shutdown_connections 2024-08-17T02:48:56.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.597+0000 7f1925000640 1 -- 172.21.15.16:0/3222957633 wait complete. 2024-08-17T02:48:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.598+0000 7f1925000640 1 Processor -- start 2024-08-17T02:48:56.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.598+0000 7f1925000640 1 -- start start 2024-08-17T02:48:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.599+0000 7f1925000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1920067de0 0x7f1920149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.599+0000 7f1925000640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19200b0830 0x7f1920098900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.599+0000 7f1925000640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19200b2ef0 0x7f1920157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.599+0000 7f1925000640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19201594e0 con 0x7f19200b2ef0 2024-08-17T02:48:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.599+0000 7f1925000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1920099330 con 0x7f1920067de0 2024-08-17T02:48:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.599+0000 7f1925000640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1920099610 con 0x7f19200b0830 2024-08-17T02:48:56.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.599+0000 7f191fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19200b2ef0 0x7f1920157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.599+0000 7f191fe00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19200b2ef0 0x7f1920157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36746/0 (socket says 172.21.15.16:36746) 2024-08-17T02:48:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.599+0000 7f191fe00640 1 -- 172.21.15.16:0/2366959885 learned_addr learned my addr 172.21.15.16:0/2366959885 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.599+0000 7f191f400640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1920067de0 0x7f1920149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.599+0000 7f1917e00640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19200b0830 0x7f1920098900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.600+0000 7f191fe00640 1 -- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19200b0830 msgr2=0x7f1920098900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.600+0000 7f191fe00640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19200b0830 0x7f1920098900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.600+0000 7f191fe00640 1 -- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1920067de0 msgr2=0x7f1920149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.604 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.600+0000 7f191fe00640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1920067de0 0x7f1920149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.600+0000 7f191fe00640 1 -- 172.21.15.16:0/2366959885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1918009ae0 con 0x7f19200b2ef0 2024-08-17T02:48:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.600+0000 7f1917e00640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f19200b0830 0x7f1920098900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.600+0000 7f191fe00640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19200b2ef0 0x7f1920157370 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f191802acd0 tx=0x7f1918034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.600+0000 7f1916a00640 1 -- 172.21.15.16:0/2366959885 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918031880 con 0x7f19200b2ef0 2024-08-17T02:48:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.600+0000 7f1916a00640 1 -- 172.21.15.16:0/2366959885 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1918007170 con 0x7f19200b2ef0 2024-08-17T02:48:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.600+0000 7f1916a00640 1 -- 172.21.15.16:0/2366959885 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191803b640 con 0x7f19200b2ef0 2024-08-17T02:48:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.600+0000 7f1925000640 1 -- 172.21.15.16:0/2366959885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19200998f0 con 0x7f19200b2ef0 2024-08-17T02:48:56.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.601+0000 7f1925000640 1 -- 172.21.15.16:0/2366959885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1920149970 con 0x7f19200b2ef0 2024-08-17T02:48:56.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.601+0000 7f1925000640 1 -- 172.21.15.16:0/2366959885 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1920063d20 con 0x7f19200b2ef0 2024-08-17T02:48:56.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.602+0000 7f1916a00640 1 -- 172.21.15.16:0/2366959885 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1918038050 con 0x7f19200b2ef0 2024-08-17T02:48:56.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.603+0000 7f1916a00640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f18fc065150 0x7f18fc067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.603+0000 7f1916a00640 1 -- 172.21.15.16:0/2366959885 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f19180a2660 con 0x7f19200b2ef0 2024-08-17T02:48:56.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.603+0000 7f1916a00640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] conn(0x7f18fc06aed0 0x7f18fc06d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.603+0000 7f1916a00640 1 -- 172.21.15.16:0/2366959885 --> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18fc06d990 con 0x7f18fc06aed0 2024-08-17T02:48:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.603+0000 7f1916a00640 1 -- 172.21.15.16:0/2366959885 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19180a4b60 con 0x7f19200b2ef0 2024-08-17T02:48:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.604+0000 7f191f400640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f18fc065150 0x7f18fc067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.604+0000 7f1917e00640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] conn(0x7f18fc06aed0 0x7f18fc06d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.604+0000 7f191f400640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f18fc065150 0x7f18fc067600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f19200ab350 tx=0x7f19100090f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.604+0000 7f1917e00640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] conn(0x7f18fc06aed0 0x7f18fc06d2e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.605+0000 7f1916a00640 1 -- 172.21.15.16:0/2366959885 <== osd.7 v2:172.21.15.142:6816/37634 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f18fc06d990 con 0x7f18fc06aed0 2024-08-17T02:48:56.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.618+0000 7f1925000640 1 -- 172.21.15.16:0/2366959885 --> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1920071050 con 0x7f18fc06aed0 2024-08-17T02:48:56.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.619+0000 7f1916a00640 1 -- 172.21.15.16:0/2366959885 <== osd.7 v2:172.21.15.142:6816/37634 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1920071050 con 0x7f18fc06aed0 2024-08-17T02:48:56.621 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-08-17T02:48:56.621 INFO:teuthology.orchestra.run.smithi016.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-08-17T02:48:56.621 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-08-17T02:48:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.619+0000 7f1925000640 1 -- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] conn(0x7f18fc06aed0 msgr2=0x7f18fc06d2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.619+0000 7f1925000640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:6816/37634,v1:172.21.15.142:6817/37634] conn(0x7f18fc06aed0 0x7f18fc06d2e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.619+0000 7f1925000640 1 -- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f18fc065150 msgr2=0x7f18fc067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.619+0000 7f1925000640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f18fc065150 0x7f18fc067600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f19200ab350 tx=0x7f19100090f0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.619+0000 7f1925000640 1 -- 172.21.15.16:0/2366959885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19200b2ef0 msgr2=0x7f1920157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.619+0000 7f1925000640 1 --2- 172.21.15.16:0/2366959885 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19200b2ef0 0x7f1920157370 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f191802acd0 tx=0x7f1918034000 comp rx=0 tx=0).stop 2024-08-17T02:48:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.619+0000 7f191fe00640 1 -- 172.21.15.16:0/2366959885 reap_dead start 2024-08-17T02:48:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.619+0000 7f1925000640 1 -- 172.21.15.16:0/2366959885 shutdown_connections 2024-08-17T02:48:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.619+0000 7f1925000640 1 -- 172.21.15.16:0/2366959885 >> 172.21.15.16:0/2366959885 conn(0x7f1920096a70 msgr2=0x7f19200af580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.619+0000 7f1925000640 1 -- 172.21.15.16:0/2366959885 shutdown_connections 2024-08-17T02:48:56.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.619+0000 7f1925000640 1 -- 172.21.15.16:0/2366959885 wait complete. 2024-08-17T02:48:56.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-08-17T02:48:56.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.786+0000 7f720c800640 1 Processor -- start 2024-08-17T02:48:56.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.786+0000 7f720c800640 1 -- start start 2024-08-17T02:48:56.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.787+0000 7f720c800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72040caa40 0x7f72040ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.787+0000 7f720c800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7204159350 0x7f72040ad380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.787+0000 7f720c800640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72040cb560 con 0x7f7204158f80 2024-08-17T02:48:56.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.787+0000 7f720c800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72040cf7e0 con 0x7f7204159350 2024-08-17T02:48:56.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.787+0000 7f720c800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72040cfac0 con 0x7f72040caa40 2024-08-17T02:48:56.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.787+0000 7f7209e00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7204158f80 0x7f72040ca330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34854/0 (socket says 172.21.15.16:34854) 2024-08-17T02:48:56.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.787+0000 7f7209e00640 1 -- 172.21.15.16:0/3691614620 learned_addr learned my addr 172.21.15.16:0/3691614620 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:48:56.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.787+0000 7f720a800640 1 --2- 172.21.15.16:0/3691614620 >> v2:172.21.15.16:3300/0 conn(0x7f7204159350 0x7f72040ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.788+0000 7f7209400640 1 --2- 172.21.15.16:0/3691614620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72040caa40 0x7f72040ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.788+0000 7f720a800640 1 -- 172.21.15.16:0/3691614620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72040caa40 msgr2=0x7f72040ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:56.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.788+0000 7f720a800640 1 --2- 172.21.15.16:0/3691614620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72040caa40 0x7f72040ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.788+0000 7f720a800640 1 -- 172.21.15.16:0/3691614620 >> v1:172.21.15.16:6789/0 conn(0x7f7204158f80 legacy=0x7f72040ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.788+0000 7f720a800640 1 -- 172.21.15.16:0/3691614620 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72040caf10 con 0x7f7204159350 2024-08-17T02:48:56.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.788+0000 7f7209400640 1 --2- 172.21.15.16:0/3691614620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72040caa40 0x7f72040ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:56.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.789+0000 7f720a800640 1 --2- 172.21.15.16:0/3691614620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7204159350 0x7f72040ad380 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7200009e60 tx=0x7f7200009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebdca7b817b21fa server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.789+0000 7f7208a00640 1 -- 172.21.15.16:0/3691614620 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200007550 con 0x7f7204159350 2024-08-17T02:48:56.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.789+0000 7f7208a00640 1 -- 172.21.15.16:0/3691614620 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720002d050 con 0x7f7204159350 2024-08-17T02:48:56.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.789+0000 7f7208a00640 1 -- 172.21.15.16:0/3691614620 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200007920 con 0x7f7204159350 2024-08-17T02:48:56.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.790+0000 7f720c800640 1 -- 172.21.15.16:0/3691614620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7204159350 msgr2=0x7f72040ad380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.790+0000 7f720c800640 1 --2- 172.21.15.16:0/3691614620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7204159350 0x7f72040ad380 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7200009e60 tx=0x7f7200009780 comp rx=0 tx=0).stop 2024-08-17T02:48:56.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.790+0000 7f720c800640 1 -- 172.21.15.16:0/3691614620 shutdown_connections 2024-08-17T02:48:56.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.790+0000 7f720c800640 1 --2- 172.21.15.16:0/3691614620 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7204159350 0x7f72040ad380 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.790+0000 7f720c800640 1 --2- 172.21.15.16:0/3691614620 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f72040caa40 0x7f72040ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.790+0000 7f720c800640 1 -- 172.21.15.16:0/3691614620 >> 172.21.15.16:0/3691614620 conn(0x7f72040cfd90 msgr2=0x7f72040d0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:56.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.790+0000 7f720c800640 1 -- 172.21.15.16:0/3691614620 shutdown_connections 2024-08-17T02:48:56.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.790+0000 7f720c800640 1 -- 172.21.15.16:0/3691614620 wait complete. 2024-08-17T02:48:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.791+0000 7f720c800640 1 Processor -- start 2024-08-17T02:48:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.791+0000 7f720c800640 1 -- start start 2024-08-17T02:48:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.792+0000 7f720c800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72040caa40 0x7f72040b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.792+0000 7f720c800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7204158f80 0x7f7204155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.792+0000 7f720c800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7204159350 0x7f720415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.792+0000 7f720c800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040b5c60 con 0x7f7204158f80 2024-08-17T02:48:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.792+0000 7f720c800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040b5980 con 0x7f72040caa40 2024-08-17T02:48:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.792+0000 7f720c800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720415e270 con 0x7f7204159350 2024-08-17T02:48:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.792+0000 7f7209400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7204158f80 0x7f7204155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.792+0000 7f7209400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7204158f80 0x7f7204155040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36764/0 (socket says 172.21.15.16:36764) 2024-08-17T02:48:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.792+0000 7f7209400640 1 -- 172.21.15.16:0/2339625777 learned_addr learned my addr 172.21.15.16:0/2339625777 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f7209e00640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72040caa40 0x7f72040b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f720a800640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7204159350 0x7f720415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f7209400640 1 -- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7204159350 msgr2=0x7f720415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f7209400640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7204159350 0x7f720415a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f7209400640 1 -- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72040caa40 msgr2=0x7f72040b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f7209400640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72040caa40 0x7f72040b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f7209400640 1 -- 172.21.15.16:0/2339625777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7200009a20 con 0x7f7204158f80 2024-08-17T02:48:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f7209e00640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72040caa40 0x7f72040b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-17T02:48:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f7209400640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7204158f80 0x7f7204155040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f71f400d810 tx=0x7f71f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f71fd600640 1 -- 172.21.15.16:0/2339625777 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f400de00 con 0x7f7204158f80 2024-08-17T02:48:56.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f71fd600640 1 -- 172.21.15.16:0/2339625777 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f40098b0 con 0x7f7204158f80 2024-08-17T02:48:56.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f71fd600640 1 -- 172.21.15.16:0/2339625777 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4009c70 con 0x7f7204158f80 2024-08-17T02:48:56.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f720c800640 1 -- 172.21.15.16:0/2339625777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720415e550 con 0x7f7204158f80 2024-08-17T02:48:56.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.793+0000 7f720c800640 1 -- 172.21.15.16:0/2339625777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7204155580 con 0x7f7204158f80 2024-08-17T02:48:56.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.795+0000 7f720c800640 1 -- 172.21.15.16:0/2339625777 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720415e0d0 con 0x7f7204158f80 2024-08-17T02:48:56.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.795+0000 7f71fd600640 1 -- 172.21.15.16:0/2339625777 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f71f4014640 con 0x7f7204158f80 2024-08-17T02:48:56.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.797+0000 7f71fd600640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f71e8065150 0x7f71e8067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.797+0000 7f71fd600640 1 -- 172.21.15.16:0/2339625777 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f71f40825f0 con 0x7f7204158f80 2024-08-17T02:48:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.797+0000 7f7209e00640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f71e8065150 0x7f71e8067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.798+0000 7f7209e00640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f71e8065150 0x7f71e8067600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f71f80017a0 tx=0x7f71f8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:56.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.801+0000 7f71fd600640 1 -- 172.21.15.16:0/2339625777 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f720415e0d0 con 0x7f7204158f80 2024-08-17T02:48:56.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.982+0000 7f720c800640 1 -- 172.21.15.16:0/2339625777 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f720415e0d0 con 0x7f71e8065150 2024-08-17T02:48:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.983+0000 7f71fd600640 1 -- 172.21.15.16:0/2339625777 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f720415e0d0 con 0x7f71e8065150 2024-08-17T02:48:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:instructed osd(s) 7 to deep-scrub 2024-08-17T02:48:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.985+0000 7f720c800640 1 -- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f71e8065150 msgr2=0x7f71e8067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.985+0000 7f720c800640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f71e8065150 0x7f71e8067600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f71f80017a0 tx=0x7f71f8009040 comp rx=0 tx=0).stop 2024-08-17T02:48:56.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.985+0000 7f720c800640 1 -- 172.21.15.16:0/2339625777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7204158f80 msgr2=0x7f7204155040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.985+0000 7f720c800640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7204158f80 0x7f7204155040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f71f400d810 tx=0x7f71f4008040 comp rx=0 tx=0).stop 2024-08-17T02:48:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.985+0000 7f720c800640 1 -- 172.21.15.16:0/2339625777 shutdown_connections 2024-08-17T02:48:56.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.985+0000 7f720c800640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f71e8065150 0x7f71e8067600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.985+0000 7f720c800640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f7204159350 0x7f720415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.985+0000 7f720c800640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7204158f80 0x7f7204155040 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.985+0000 7f720c800640 1 --2- 172.21.15.16:0/2339625777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72040caa40 0x7f72040b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:56.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.985+0000 7f720c800640 1 -- 172.21.15.16:0/2339625777 >> 172.21.15.16:0/2339625777 conn(0x7f72040cfd90 msgr2=0x7f72040c5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:56.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.985+0000 7f720c800640 1 -- 172.21.15.16:0/2339625777 shutdown_connections 2024-08-17T02:48:56.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:56.985+0000 7f720c800640 1 -- 172.21.15.16:0/2339625777 wait complete. 2024-08-17T02:48:57.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:48:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.151+0000 7fbfdb200640 1 Processor -- start 2024-08-17T02:48:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.152+0000 7fbfdb200640 1 -- start start 2024-08-17T02:48:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.153+0000 7fbfdb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfdc155900 0x7fbfdc155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.153+0000 7fbfdb200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbfdc0b0270 0x7fbfdc156210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.153+0000 7fbfdb200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfdc157690 con 0x7fbfdc157b20 2024-08-17T02:48:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.153+0000 7fbfdb200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfdc156a80 con 0x7fbfdc0b0270 2024-08-17T02:48:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.153+0000 7fbfdb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc156d60 con 0x7fbfdc155900 2024-08-17T02:48:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.153+0000 7fbfd3e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbfdc0b0270 0x7fbfdc156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.153+0000 7fbfd3e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbfdc0b0270 0x7fbfdc156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36770/0 (socket says 172.21.15.16:36770) 2024-08-17T02:48:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.153+0000 7fbfd3e00640 1 -- 172.21.15.16:0/2151481989 learned_addr learned my addr 172.21.15.16:0/2151481989 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.153+0000 7fbfda800640 1 --1- 172.21.15.16:0/2151481989 >> v1:172.21.15.16:6789/0 conn(0x7fbfdc157b20 0x7fbfdc157fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34864/0 (socket says 172.21.15.16:34864) 2024-08-17T02:48:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.153+0000 7fbfd9e00640 1 --2- 172.21.15.16:0/2151481989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfdc155900 0x7fbfdc155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.154+0000 7fbfd3e00640 1 -- 172.21.15.16:0/2151481989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfdc155900 msgr2=0x7fbfdc155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:48:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.154+0000 7fbfd3e00640 1 --2- 172.21.15.16:0/2151481989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfdc155900 0x7fbfdc155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.154+0000 7fbfd3e00640 1 -- 172.21.15.16:0/2151481989 >> v1:172.21.15.16:6789/0 conn(0x7fbfdc157b20 legacy=0x7fbfdc157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.154+0000 7fbfd3e00640 1 -- 172.21.15.16:0/2151481989 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc157040 con 0x7fbfdc0b0270 2024-08-17T02:48:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.155+0000 7fbfd3e00640 1 --2- 172.21.15.16:0/2151481989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfdc0b0270 0x7fbfdc156210 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009910 tx=0x7fbfc40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb800244b463a54 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.155+0000 7fbfd9400640 1 -- 172.21.15.16:0/2151481989 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4031510 con 0x7fbfdc0b0270 2024-08-17T02:48:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.155+0000 7fbfd9400640 1 -- 172.21.15.16:0/2151481989 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc402f050 con 0x7fbfdc0b0270 2024-08-17T02:48:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.155+0000 7fbfd9400640 1 -- 172.21.15.16:0/2151481989 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc40318e0 con 0x7fbfdc0b0270 2024-08-17T02:48:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.155+0000 7fbfdb200640 1 -- 172.21.15.16:0/2151481989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfdc0b0270 msgr2=0x7fbfdc156210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.155+0000 7fbfdb200640 1 --2- 172.21.15.16:0/2151481989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfdc0b0270 0x7fbfdc156210 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009910 tx=0x7fbfc40091d0 comp rx=0 tx=0).stop 2024-08-17T02:48:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.156+0000 7fbfdb200640 1 -- 172.21.15.16:0/2151481989 shutdown_connections 2024-08-17T02:48:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.156+0000 7fbfdb200640 1 --2- 172.21.15.16:0/2151481989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfdc0b0270 0x7fbfdc156210 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.156+0000 7fbfdb200640 1 --2- 172.21.15.16:0/2151481989 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfdc155900 0x7fbfdc155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.156+0000 7fbfdb200640 1 -- 172.21.15.16:0/2151481989 >> 172.21.15.16:0/2151481989 conn(0x7fbfdc067750 msgr2=0x7fbfdc0cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.156+0000 7fbfdb200640 1 -- 172.21.15.16:0/2151481989 shutdown_connections 2024-08-17T02:48:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.156+0000 7fbfdb200640 1 -- 172.21.15.16:0/2151481989 wait complete. 2024-08-17T02:48:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.157+0000 7fbfdb200640 1 Processor -- start 2024-08-17T02:48:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.157+0000 7fbfdb200640 1 -- start start 2024-08-17T02:48:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.158+0000 7fbfdb200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfdc0b0270 0x7fbfdc149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.158+0000 7fbfdb200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfdc155900 0x7fbfdc068540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.158+0000 7fbfdb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfdc157b20 0x7fbfdc068a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.158+0000 7fbfdb200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc16d2c0 con 0x7fbfdc0b0270 2024-08-17T02:48:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.158+0000 7fbfdb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc16c6b0 con 0x7fbfdc157b20 2024-08-17T02:48:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.158+0000 7fbfdb200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfdc16c990 con 0x7fbfdc155900 2024-08-17T02:48:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.158+0000 7fbfd9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfdc0b0270 0x7fbfdc149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.158+0000 7fbfd9e00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfdc0b0270 0x7fbfdc149610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36780/0 (socket says 172.21.15.16:36780) 2024-08-17T02:48:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.158+0000 7fbfd9e00640 1 -- 172.21.15.16:0/1968558375 learned_addr learned my addr 172.21.15.16:0/1968558375 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:48:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.158+0000 7fbfd3e00640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfdc155900 0x7fbfdc068540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.158+0000 7fbfda800640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfdc157b20 0x7fbfdc068a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.159+0000 7fbfd9e00640 1 -- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfdc155900 msgr2=0x7fbfdc068540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.159+0000 7fbfd9e00640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfdc155900 0x7fbfdc068540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.159+0000 7fbfd9e00640 1 -- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfdc157b20 msgr2=0x7fbfdc068a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.159+0000 7fbfd9e00640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfdc157b20 0x7fbfdc068a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.159+0000 7fbfd9e00640 1 -- 172.21.15.16:0/1968558375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc4009450 con 0x7fbfdc0b0270 2024-08-17T02:48:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.159+0000 7fbfd9e00640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfdc0b0270 0x7fbfdc149610 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00d890 tx=0x7fbfcc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.159+0000 7fbfd1600640 1 -- 172.21.15.16:0/1968558375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00de20 con 0x7fbfdc0b0270 2024-08-17T02:48:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.159+0000 7fbfd1600640 1 -- 172.21.15.16:0/1968558375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc00b8d0 con 0x7fbfdc0b0270 2024-08-17T02:48:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.159+0000 7fbfdb200640 1 -- 172.21.15.16:0/1968558375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfdc16cc70 con 0x7fbfdc0b0270 2024-08-17T02:48:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.159+0000 7fbfd1600640 1 -- 172.21.15.16:0/1968558375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc00bc70 con 0x7fbfdc0b0270 2024-08-17T02:48:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.159+0000 7fbfdb200640 1 -- 172.21.15.16:0/1968558375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfdc149b50 con 0x7fbfdc0b0270 2024-08-17T02:48:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.161+0000 7fbfdb200640 1 -- 172.21.15.16:0/1968558375 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfdc063840 con 0x7fbfdc0b0270 2024-08-17T02:48:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.161+0000 7fbfd1600640 1 -- 172.21.15.16:0/1968558375 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fbfcc014640 con 0x7fbfdc0b0270 2024-08-17T02:48:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.162+0000 7fbfd1600640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbfb8065150 0x7fbfb8067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:48:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.162+0000 7fbfd1600640 1 -- 172.21.15.16:0/1968558375 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fbfcc082770 con 0x7fbfdc0b0270 2024-08-17T02:48:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.163+0000 7fbfd3e00640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbfb8065150 0x7fbfb8067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:48:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.163+0000 7fbfd3e00640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbfb8065150 0x7fbfb8067600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4036000 tx=0x7fbfc4037000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:48:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.166+0000 7fbfd1600640 1 -- 172.21.15.16:0/1968558375 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fbfcc050b90 con 0x7fbfdc0b0270 2024-08-17T02:48:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.346+0000 7fbfdb200640 1 -- 172.21.15.16:0/1968558375 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfdc16d400 con 0x7fbfb8065150 2024-08-17T02:48:57.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.349+0000 7fbfd1600640 1 -- 172.21.15.16:0/1968558375 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50620 (secure 0 0 0) 0x7fbfc4042070 con 0x7fbfb8065150 2024-08-17T02:48:57.351 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:48:57.351 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-17T02:48:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.351+0000 7fbfdb200640 1 -- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbfb8065150 msgr2=0x7fbfb8067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.351+0000 7fbfdb200640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbfb8065150 0x7fbfb8067600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4036000 tx=0x7fbfc4037000 comp rx=0 tx=0).stop 2024-08-17T02:48:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.351+0000 7fbfdb200640 1 -- 172.21.15.16:0/1968558375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfdc0b0270 msgr2=0x7fbfdc149610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:48:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.351+0000 7fbfdb200640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfdc0b0270 0x7fbfdc149610 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00d890 tx=0x7fbfcc008040 comp rx=0 tx=0).stop 2024-08-17T02:48:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.351+0000 7fbfdb200640 1 -- 172.21.15.16:0/1968558375 shutdown_connections 2024-08-17T02:48:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.351+0000 7fbfdb200640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fbfb8065150 0x7fbfb8067600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.351+0000 7fbfdb200640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfdc157b20 0x7fbfdc068a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.351+0000 7fbfdb200640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fbfdc155900 0x7fbfdc068540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.351+0000 7fbfdb200640 1 --2- 172.21.15.16:0/1968558375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfdc0b0270 0x7fbfdc149610 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:48:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.351+0000 7fbfdb200640 1 -- 172.21.15.16:0/1968558375 >> 172.21.15.16:0/1968558375 conn(0x7fbfdc067750 msgr2=0x7fbfdc14b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:48:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.351+0000 7fbfdb200640 1 -- 172.21.15.16:0/1968558375 shutdown_connections 2024-08-17T02:48:57.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:48:57.351+0000 7fbfdb200640 1 -- 172.21.15.16:0/1968558375 wait complete. 2024-08-17T02:48:57.367 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-08-17T02:48:56.292643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2400956,"kb_used_data":5448,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752573764,"statfs":{"total":773094113280,"available":770635534336,"internally_reserved":0,"allocated":5578752,"data_stored":3037136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001987"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-08-17T02:48:45.803029+0000","last_change":"2024-08-17T02:45:16.001117+0000","last_active":"2024-08-17T02:48:45.803029+0000","last_peered":"2024-08-17T02:48:45.803029+0000","last_clean":"2024-08-17T02:48:45.803029+0000","last_became_active":"2024-08-17T02:45:16.000788+0000","last_became_peered":"2024-08-17T02:45:16.000788+0000","last_unstale":"2024-08-17T02:48:45.803029+0000","last_undegraded":"2024-08-17T02:48:45.803029+0000","last_fullsized":"2024-08-17T02:48:45.803029+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-17T02:45:14.986707+0000","last_clean_scrub_stamp":"2024-08-17T02:45:14.986707+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T06:27:37.295374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.81799999999999995},"min":{"1min":0.61499999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90900000000000003,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.53100000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.88900000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.59499999999999997}]},{"osd":1,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.64200000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96399999999999997,"5min":2.202,"15min":2.202},"last":0.755},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.65900000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.877,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.622}]},{"osd":2,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81899999999999995,"15min":0.84699999999999998},"min":{"1min":0.57799999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94399999999999995,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.90900000000000003,"15min":0.95499999999999996},"min":{"1min":0.69799999999999995,"5min":0.621,"15min":0.621},"max":{"1min":0.93400000000000005,"5min":4.6870000000000003,"15min":4.6870000000000003},"last":0.64000000000000001}]},{"osd":3,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76100000000000001,"15min":0.78000000000000003},"min":{"1min":0.57699999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":2.798,"15min":2.798},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72799999999999998,"15min":0.73299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89500000000000002,"5min":1.081,"15min":1.081},"last":0.54900000000000004}]},{"osd":4,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.67800000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.505},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73999999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.48199999999999998}]},{"osd":5,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.66000000000000003},"min":{"1min":0.504,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67500000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79000000000000004,"5min":1.337,"15min":1.337},"last":0.56899999999999995}]},{"osd":6,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84599999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79900000000000004,"5min":1.052,"15min":1.052},"last":0.53400000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73899999999999999,"15min":0.70099999999999996},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98599999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71699999999999997,"15min":0.65900000000000003},"min":{"1min":0.60699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.65800000000000003}]},{"osd":1,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.58599999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":0.751},"min":{"1min":0.61799999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.998,"5min":5.0970000000000004,"15min":5.0970000000000004},"last":0.70599999999999996}]},{"osd":2,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66800000000000004,"15min":0.64800000000000002},"min":{"1min":0.51500000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75,"15min":0.69899999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":5.0579999999999998,"15min":5.0579999999999998},"last":0.64900000000000002}]},{"osd":3,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.85099999999999998,"15min":0.84399999999999997},"min":{"1min":0.60799999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93100000000000005,"5min":5.1849999999999996,"15min":5.1849999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.76100000000000001},"min":{"1min":0.60699999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.052,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77700000000000002,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84399999999999997,"5min":5.7030000000000003,"15min":5.7030000000000003},"last":0.42899999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.47899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.82199999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.74399999999999999,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94999999999999996,"5min":5.1639999999999997,"15min":5.1639999999999997},"last":0.67900000000000005}]},{"osd":7,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.73999999999999999},"min":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91500000000000004,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.57399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.98899999999999999,"5min":1.405,"15min":1.405},"last":0.57699999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298476,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073364,"statfs":{"total":96636764160,"available":96331124736,"internally_reserved":0,"allocated":1142784,"data_stored":822418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.47699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.58299999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.155,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.59899999999999998}]},{"osd":1,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68400000000000005,"15min":0.65200000000000002},"min":{"1min":0.57799999999999996,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91100000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67800000000000005,"15min":0.64400000000000002},"min":{"1min":0.52100000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.57799999999999996}]},{"osd":2,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.67000000000000004,"15min":0.64600000000000002},"min":{"1min":0.59499999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93100000000000005,"5min":0.999,"15min":0.999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.60299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.748}]},{"osd":3,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.624},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.68200000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.202,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.43099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65700000000000003}]},{"osd":6,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.48499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.68500000000000005},"min":{"1min":0.48999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.876,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.84299999999999997}]},{"osd":7,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.70099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64800000000000002,"15min":0.61799999999999999},"min":{"1min":0.34100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.86899999999999999,"5min":1.073,"15min":1.073},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65400000000000003,"15min":0.64100000000000001},"min":{"1min":0.53400000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56599999999999995}]},{"osd":1,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.624,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86599999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96499999999999997,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.68700000000000006},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68100000000000005,"15min":0.67400000000000004},"min":{"1min":0.52000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.82899999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.73699999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.55600000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81100000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73099999999999998,"15min":0.71999999999999997},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94299999999999995,"5min":1.147,"15min":1.147},"last":0.59599999999999997}]},{"osd":5,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67400000000000004,"15min":0.69299999999999995},"min":{"1min":0.40300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78300000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66900000000000004,"15min":0.67800000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84799999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65100000000000002}]},{"osd":6,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.45200000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67300000000000004,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78800000000000003,"5min":1.2,"15min":1.2},"last":0.76200000000000001}]},{"osd":7,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70699999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.504,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67700000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302052,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069788,"statfs":{"total":96636764160,"available":96327462912,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.48699999999999999,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.627,"5min":1.901,"15min":1.901},"last":0.374},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59699999999999998,"15min":0.63300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":3.552,"15min":3.552},"last":0.48699999999999999}]},{"osd":1,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.33200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.67200000000000004,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73499999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.41099999999999998}]},{"osd":2,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.64700000000000002,"15min":0.69799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73599999999999999,"5min":3.8149999999999999,"15min":3.8149999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.625,"15min":0.63900000000000001},"min":{"1min":0.502,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73399999999999999,"5min":1.974,"15min":1.974},"last":0.51000000000000001}]},{"osd":4,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.266,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.89600000000000002,"15min":0.94999999999999996},"min":{"1min":0.56999999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.206,"5min":4.3360000000000003,"15min":4.3360000000000003},"last":0.69399999999999995}]},{"osd":5,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.84699999999999998,"15min":0.874},"min":{"1min":0.60199999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0860000000000001,"5min":3.371,"15min":3.371},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.873,"15min":0.89900000000000002},"min":{"1min":0.59399999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1679999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.77700000000000002}]},{"osd":6,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.91000000000000003,"15min":0.96099999999999997},"min":{"1min":0.58299999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.3109999999999999,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.91700000000000004},"min":{"1min":0.72699999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2350000000000001,"5min":3.331,"15min":3.331},"last":0.74299999999999999}]},{"osd":7,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.94099999999999995,"15min":0.98299999999999998},"min":{"1min":0.66500000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.105,"5min":3.931,"15min":3.931},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.95699999999999996,"15min":1.0029999999999999},"min":{"1min":0.67600000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4279999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.72099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302056,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069784,"statfs":{"total":96636764160,"available":96327458816,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.443,"15min":0.45900000000000002},"min":{"1min":0.314,"5min":0.255,"15min":0.255},"max":{"1min":0.58399999999999996,"5min":1.843,"15min":1.843},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52600000000000002,"15min":0.54800000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58599999999999997,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.67100000000000004}]},{"osd":1,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.73499999999999999,"5min":1.903,"15min":1.903},"last":0.377},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.61499999999999999,"15min":0.66200000000000003},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.60299999999999998,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.54000000000000004}]},{"osd":3,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.72399999999999998,"15min":0.81899999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68799999999999994,"5min":7.5960000000000001,"15min":7.5960000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.77000000000000002,"15min":0.83999999999999997},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82299999999999995,"5min":6.0499999999999998,"15min":6.0499999999999998},"last":0.64200000000000002}]},{"osd":4,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.85399999999999998},"min":{"1min":0.72899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.97599999999999998,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":1.0069999999999999,"15min":1.099},"min":{"1min":0.60999999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0860000000000001,"5min":5.7009999999999996,"15min":5.7009999999999996},"last":0.74099999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.96199999999999997,"15min":1.0569999999999999},"min":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.006,"5min":7.5220000000000002,"15min":7.5220000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.91200000000000003,"15min":0.97099999999999997},"min":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91900000000000004,"5min":5.968,"15min":5.968},"last":0.72199999999999998}]},{"osd":6,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":1.012,"15min":1.1100000000000001},"min":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99399999999999999,"5min":5.8780000000000001,"15min":5.8780000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.96999999999999997,"15min":1.0600000000000001},"min":{"1min":0.64500000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0009999999999999,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.77500000000000002}]},{"osd":7,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":1.03,"15min":1.121},"min":{"1min":0.70699999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.054,"5min":5.9119999999999999,"15min":5.9119999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0189999999999999,"15min":1.113},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.95799999999999996,"5min":7.4800000000000004,"15min":7.4800000000000004},"last":0.81999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302632,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069208,"statfs":{"total":96636764160,"available":96326868992,"internally_reserved":0,"allocated":1142784,"data_stored":822418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.316,"5min":0.255,"15min":0.255},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.57199999999999995,"15min":0.59999999999999998},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.70699999999999996,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.53200000000000003}]},{"osd":2,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69199999999999995,"5min":2.165,"15min":2.165},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.40100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.64500000000000002,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.433}]},{"osd":3,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.47899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82299999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.64100000000000001},"min":{"1min":0.41599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.85899999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.45700000000000002}]},{"osd":4,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.66400000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0229999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.748},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.66000000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.79000000000000004}]},{"osd":5,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.622,"5min":0.501,"15min":0.501},"max":{"1min":1.034,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.68500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.085,"5min":1.131,"15min":1.131},"last":0.82099999999999995}]},{"osd":6,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.65000000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.1060000000000001,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":0.90100000000000002}]},{"osd":7,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.85599999999999998,"15min":0.872},"min":{"1min":0.67000000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0700000000000001,"5min":2.577,"15min":2.577},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.63500000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.026,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.91400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302052,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069788,"statfs":{"total":96636764160,"available":96327462912,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.317,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.46899999999999997,"15min":0.46100000000000002},"min":{"1min":0.40000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67400000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.36699999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70999999999999996,"5min":1.105,"15min":1.105},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.44600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.871,"15min":0.871},"last":0.42299999999999999}]},{"osd":3,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.46100000000000002},"min":{"1min":0.39200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.754,"5min":1.181,"15min":1.181},"last":0.80000000000000004}]},{"osd":4,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78400000000000003,"15min":0.77900000000000003},"min":{"1min":0.67600000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.093,"5min":1.468,"15min":1.468},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.71899999999999997,"15min":0.70799999999999996},"min":{"1min":0.56499999999999995,"5min":0.44,"15min":0.44},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.56399999999999995}]},{"osd":5,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.59399999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75600000000000001,"15min":0.753},"min":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.91700000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.73599999999999999}]},{"osd":6,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96599999999999997,"5min":1.444,"15min":1.444},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80600000000000005,"15min":0.81499999999999995},"min":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.01,"5min":1.756,"15min":1.756},"last":0.78800000000000003}]},{"osd":7,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81899999999999995,"15min":0.81999999999999995},"min":{"1min":0.67800000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1339999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.65800000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99299999999999999,"5min":1.298,"15min":1.298},"last":0.63300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:48:57.375 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-08-17T02:45:14.986707+0000 time.struct_time(tm_year=2024, tm_mon=8, tm_mday=17, tm_hour=2, tm_min=45, tm_sec=14, tm_wday=5, tm_yday=230, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=8, tm_mday=17, tm_hour=2, tm_min=48, tm_sec=51, tm_wday=5, tm_yday=230, tm_isdst=0) 2024-08-17T02:48:57.375 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-08-17T02:49:17.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-17T02:49:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.537+0000 7fde20200640 1 Processor -- start 2024-08-17T02:49:17.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.537+0000 7fde20200640 1 -- start start 2024-08-17T02:49:17.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.538+0000 7fde20200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde180a98e0 0x7fde180ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.538+0000 7fde20200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde180aaa40 con 0x7fde180ad070 2024-08-17T02:49:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.538+0000 7fde20200640 1 -- --> v1:172.21.15.142:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde180a9e30 con 0x7fde18158b10 2024-08-17T02:49:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.538+0000 7fde20200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde180abb20 con 0x7fde180a98e0 2024-08-17T02:49:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.539+0000 7fde1ce00640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fde180ad070 0x7fde180aaed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49700/0 (socket says 172.21.15.16:49700) 2024-08-17T02:49:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.539+0000 7fde1e200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde180a98e0 0x7fde180ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:17.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.539+0000 7fde1ce00640 1 -- 172.21.15.16:0/2864731133 learned_addr learned my addr 172.21.15.16:0/2864731133 (peer_addr_for_me v1:172.21.15.16:0/0) 2024-08-17T02:49:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.539+0000 7fde1e200640 1 -- 172.21.15.16:0/2864731133 >> v1:172.21.15.142:6789/0 conn(0x7fde18158b10 legacy=0x7fde180c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.539+0000 7fde1e200640 1 -- 172.21.15.16:0/2864731133 >> v1:172.21.15.16:6789/0 conn(0x7fde180ad070 legacy=0x7fde180aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.539+0000 7fde1e200640 1 -- 172.21.15.16:0/2864731133 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde180aa110 con 0x7fde180a98e0 2024-08-17T02:49:17.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.540+0000 7fde1e200640 1 --2- 172.21.15.16:0/2864731133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde180a98e0 0x7fde180ab5e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fde10009e60 tx=0x7fde10009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad42aabb32c7205 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:49:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.540+0000 7fde17e00640 1 -- 172.21.15.16:0/2864731133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10007550 con 0x7fde180a98e0 2024-08-17T02:49:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.540+0000 7fde17e00640 1 -- 172.21.15.16:0/2864731133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1002d050 con 0x7fde180a98e0 2024-08-17T02:49:17.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.540+0000 7fde17e00640 1 -- 172.21.15.16:0/2864731133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde10007920 con 0x7fde180a98e0 2024-08-17T02:49:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.541+0000 7fde20200640 1 -- 172.21.15.16:0/2864731133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde180a98e0 msgr2=0x7fde180ab5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.541+0000 7fde20200640 1 --2- 172.21.15.16:0/2864731133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde180a98e0 0x7fde180ab5e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fde10009e60 tx=0x7fde10009780 comp rx=0 tx=0).stop 2024-08-17T02:49:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.541+0000 7fde20200640 1 -- 172.21.15.16:0/2864731133 shutdown_connections 2024-08-17T02:49:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.541+0000 7fde20200640 1 --2- 172.21.15.16:0/2864731133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde180a98e0 0x7fde180ab5e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.541+0000 7fde20200640 1 -- 172.21.15.16:0/2864731133 >> 172.21.15.16:0/2864731133 conn(0x7fde18066cd0 msgr2=0x7fde180670d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:49:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.541+0000 7fde20200640 1 -- 172.21.15.16:0/2864731133 shutdown_connections 2024-08-17T02:49:17.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.541+0000 7fde20200640 1 -- 172.21.15.16:0/2864731133 wait complete. 2024-08-17T02:49:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.542+0000 7fde20200640 1 Processor -- start 2024-08-17T02:49:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.543+0000 7fde20200640 1 -- start start 2024-08-17T02:49:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.543+0000 7fde20200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde180a98e0 0x7fde18140e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.543+0000 7fde20200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde180ad070 0x7fde1815c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.543+0000 7fde20200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde18158b10 0x7fde1815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.543+0000 7fde20200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1815c230 con 0x7fde180ad070 2024-08-17T02:49:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.543+0000 7fde20200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1815b620 con 0x7fde180a98e0 2024-08-17T02:49:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.543+0000 7fde20200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1815b900 con 0x7fde18158b10 2024-08-17T02:49:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.544+0000 7fde1ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde180ad070 0x7fde1815c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.544+0000 7fde1ce00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde180ad070 0x7fde1815c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48592/0 (socket says 172.21.15.16:48592) 2024-08-17T02:49:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.544+0000 7fde1ce00640 1 -- 172.21.15.16:0/373947933 learned_addr learned my addr 172.21.15.16:0/373947933 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:49:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.544+0000 7fde1e200640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde18158b10 0x7fde1815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.544+0000 7fde1d800640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde180a98e0 0x7fde18140e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.544+0000 7fde1ce00640 1 -- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde18158b10 msgr2=0x7fde1815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.544+0000 7fde1ce00640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde18158b10 0x7fde1815ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.544+0000 7fde1ce00640 1 -- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde180a98e0 msgr2=0x7fde18140e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.544+0000 7fde1ce00640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde180a98e0 0x7fde18140e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.544+0000 7fde1ce00640 1 -- 172.21.15.16:0/373947933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde10009a20 con 0x7fde180ad070 2024-08-17T02:49:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.545+0000 7fde1ce00640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde180ad070 0x7fde1815c6c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fde0400c590 tx=0x7fde04007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:49:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.545+0000 7fde14c00640 1 -- 172.21.15.16:0/373947933 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04005aa0 con 0x7fde180ad070 2024-08-17T02:49:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.545+0000 7fde14c00640 1 -- 172.21.15.16:0/373947933 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde04005c20 con 0x7fde180ad070 2024-08-17T02:49:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.545+0000 7fde14c00640 1 -- 172.21.15.16:0/373947933 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde040070f0 con 0x7fde180ad070 2024-08-17T02:49:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.545+0000 7fde20200640 1 -- 172.21.15.16:0/373947933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1815bbe0 con 0x7fde180ad070 2024-08-17T02:49:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.545+0000 7fde20200640 1 -- 172.21.15.16:0/373947933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1815cc00 con 0x7fde180ad070 2024-08-17T02:49:17.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.546+0000 7fde20200640 1 -- 172.21.15.16:0/373947933 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1815c3a0 con 0x7fde180ad070 2024-08-17T02:49:17.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.547+0000 7fde14c00640 1 -- 172.21.15.16:0/373947933 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7fde04007540 con 0x7fde180ad070 2024-08-17T02:49:17.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.548+0000 7fde14c00640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fddf8065150 0x7fddf8067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:17.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.548+0000 7fde14c00640 1 -- 172.21.15.16:0/373947933 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7fde040812f0 con 0x7fde180ad070 2024-08-17T02:49:17.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.549+0000 7fde1d800640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fddf8065150 0x7fddf8067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.549+0000 7fde1d800640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fddf8065150 0x7fddf8067600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fde0c003720 tx=0x7fde0c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:49:17.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.553+0000 7fde14c00640 1 -- 172.21.15.16:0/373947933 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7fde0404f630 con 0x7fde180ad070 2024-08-17T02:49:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.732+0000 7fde20200640 1 -- 172.21.15.16:0/373947933 --> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde1815c060 con 0x7fddf8065150 2024-08-17T02:49:17.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.734+0000 7fde14c00640 1 -- 172.21.15.16:0/373947933 <== mgr.4102 v2:172.21.15.142:6832/37028 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50566 (secure 0 0 0) 0x7fde1815c060 con 0x7fddf8065150 2024-08-17T02:49:17.736 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:49:17.736 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-08-17T02:49:17.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.736+0000 7fde20200640 1 -- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fddf8065150 msgr2=0x7fddf8067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:17.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.736+0000 7fde20200640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fddf8065150 0x7fddf8067600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fde0c003720 tx=0x7fde0c009040 comp rx=0 tx=0).stop 2024-08-17T02:49:17.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.736+0000 7fde20200640 1 -- 172.21.15.16:0/373947933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde180ad070 msgr2=0x7fde1815c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:17.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.736+0000 7fde20200640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde180ad070 0x7fde1815c6c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fde0400c590 tx=0x7fde04007ee0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.737+0000 7fde20200640 1 -- 172.21.15.16:0/373947933 shutdown_connections 2024-08-17T02:49:17.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.737+0000 7fde20200640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7fddf8065150 0x7fddf8067600 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.737+0000 7fde20200640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7fde18158b10 0x7fde1815ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.737+0000 7fde20200640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde180ad070 0x7fde1815c6c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.737+0000 7fde20200640 1 --2- 172.21.15.16:0/373947933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde180a98e0 0x7fde18140e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.737+0000 7fde20200640 1 -- 172.21.15.16:0/373947933 >> 172.21.15.16:0/373947933 conn(0x7fde18066cd0 msgr2=0x7fde180af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:49:17.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.737+0000 7fde20200640 1 -- 172.21.15.16:0/373947933 shutdown_connections 2024-08-17T02:49:17.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.737+0000 7fde20200640 1 -- 172.21.15.16:0/373947933 wait complete. 2024-08-17T02:49:17.753 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-08-17T02:49:16.296977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2400944,"kb_used_data":5448,"kb_used_omap":13,"kb_used_meta":2395378,"kb_avail":752573776,"statfs":{"total":773094113280,"available":770635546624,"internally_reserved":0,"allocated":5578752,"data_stored":3037136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2452867776},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002527"},"pg_stats":[{"pgid":"1.0","version":"46'119","reported_seq":199,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-08-17T02:48:56.018689+0000","last_change":"2024-08-17T02:48:56.018659+0000","last_active":"2024-08-17T02:48:56.018689+0000","last_peered":"2024-08-17T02:48:56.018689+0000","last_clean":"2024-08-17T02:48:56.018689+0000","last_became_active":"2024-08-17T02:45:16.000788+0000","last_became_peered":"2024-08-17T02:45:16.000788+0000","last_unstale":"2024-08-17T02:48:56.018689+0000","last_undegraded":"2024-08-17T02:48:56.018689+0000","last_fullsized":"2024-08-17T02:48:56.018689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'119","last_scrub_stamp":"2024-08-17T02:48:56.018585+0000","last_deep_scrub":"46'119","last_deep_scrub_stamp":"2024-08-17T02:48:56.018585+0000","last_clean_scrub_stamp":"2024-08-17T02:48:56.018585+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-08-18T13:07:16.572346+0000","scrub_duration":0.0062500660000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.81799999999999995},"min":{"1min":0.61499999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90900000000000003,"5min":3.8780000000000001,"15min":3.8780000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.67500000000000004},"min":{"1min":0.53100000000000003,"5min":0.438,"15min":0.438},"max":{"1min":0.88900000000000001,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.68300000000000005}]},{"osd":1,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.64200000000000002,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96399999999999997,"5min":2.202,"15min":2.202},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78900000000000003,"15min":0.80400000000000005},"min":{"1min":0.65900000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.877,"5min":2.8769999999999998,"15min":2.8769999999999998},"last":0.66000000000000003}]},{"osd":2,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.81899999999999995,"15min":0.84699999999999998},"min":{"1min":0.57799999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94399999999999995,"5min":4.4530000000000003,"15min":4.4530000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.90900000000000003,"15min":0.95499999999999996},"min":{"1min":0.69799999999999995,"5min":0.621,"15min":0.621},"max":{"1min":0.93400000000000005,"5min":4.6870000000000003,"15min":4.6870000000000003},"last":0.79900000000000004}]},{"osd":3,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76100000000000001,"15min":0.78000000000000003},"min":{"1min":0.57699999999999996,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91500000000000004,"5min":2.798,"15min":2.798},"last":0.62},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72799999999999998,"15min":0.73299999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89500000000000002,"5min":1.081,"15min":1.081},"last":0.84099999999999997}]},{"osd":4,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.67800000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73999999999999999,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.72599999999999998}]},{"osd":5,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.66000000000000003},"min":{"1min":0.504,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67500000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.87},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66600000000000004,"15min":0.68200000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79000000000000004,"5min":1.337,"15min":1.337},"last":0.78200000000000003}]},{"osd":6,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.84599999999999997,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.79900000000000004,"5min":1.052,"15min":1.052},"last":0.88200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.73899999999999999,"15min":0.70099999999999996},"min":{"1min":0.45400000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98599999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71699999999999997,"15min":0.65900000000000003},"min":{"1min":0.60699999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.91700000000000004,"5min":5.1280000000000001,"15min":5.1280000000000001},"last":0.75900000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.74399999999999999,"15min":0.73099999999999998},"min":{"1min":0.58599999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":0.751},"min":{"1min":0.61799999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.998,"5min":5.0970000000000004,"15min":5.0970000000000004},"last":0.68799999999999994}]},{"osd":2,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.66800000000000004,"15min":0.64800000000000002},"min":{"1min":0.51500000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75,"15min":0.69899999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.95899999999999996,"5min":5.0579999999999998,"15min":5.0579999999999998},"last":0.52000000000000002}]},{"osd":3,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.85099999999999998,"15min":0.84399999999999997},"min":{"1min":0.60799999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93100000000000005,"5min":5.1849999999999996,"15min":5.1849999999999996},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.76400000000000001,"15min":0.76100000000000001},"min":{"1min":0.60699999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.052,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.66100000000000003,"15min":0.63800000000000001},"min":{"1min":0.39600000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77700000000000002,"5min":4.9770000000000003,"15min":4.9770000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.46500000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84399999999999997,"5min":5.7030000000000003,"15min":5.7030000000000003},"last":0.59399999999999997}]},{"osd":5,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.47899999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.82199999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.74399999999999999,"15min":0.71599999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94999999999999996,"5min":5.1639999999999997,"15min":5.1639999999999997},"last":0.628}]},{"osd":7,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.73999999999999999},"min":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91500000000000004,"5min":5.4669999999999996,"15min":5.4669999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.745,"15min":0.74399999999999999},"min":{"1min":0.57399999999999995,"5min":0.502,"15min":0.502},"max":{"1min":0.98899999999999999,"5min":1.405,"15min":1.405},"last":0.58299999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298468,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073372,"statfs":{"total":96636764160,"available":96331132928,"internally_reserved":0,"allocated":1142784,"data_stored":822418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.47699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.58299999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.155,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.57399999999999995}]},{"osd":1,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.68400000000000005,"15min":0.65200000000000002},"min":{"1min":0.57799999999999996,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91100000000000003,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67800000000000005,"15min":0.64400000000000002},"min":{"1min":0.52100000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.66100000000000003}]},{"osd":2,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.67000000000000004,"15min":0.64600000000000002},"min":{"1min":0.59499999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93100000000000005,"5min":0.999,"15min":0.999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.60299999999999998,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.55600000000000005}]},{"osd":3,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.68200000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.202,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.64800000000000002}]},{"osd":4,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59599999999999997,"15min":0.59499999999999997},"min":{"1min":0.43099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.434},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83099999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.70599999999999996}]},{"osd":6,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.48499999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80900000000000005,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67400000000000004,"15min":0.68500000000000005},"min":{"1min":0.48999999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.876,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.496}]},{"osd":7,"last update":"Sat Aug 17 02:48:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65400000000000003,"15min":0.65000000000000002},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67900000000000005,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297896,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.64800000000000002,"15min":0.61799999999999999},"min":{"1min":0.34100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.86899999999999999,"5min":1.073,"15min":1.073},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.65400000000000003,"15min":0.64100000000000001},"min":{"1min":0.53400000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.72199999999999998}]},{"osd":1,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67600000000000005,"15min":0.66000000000000003},"min":{"1min":0.624,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.86599999999999999,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71099999999999997,"15min":0.70099999999999996},"min":{"1min":0.47599999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.96499999999999997,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.76300000000000001}]},{"osd":2,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.70299999999999996,"15min":0.68700000000000006},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68100000000000005,"15min":0.67400000000000004},"min":{"1min":0.52000000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.82899999999999996,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.55600000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81100000000000005,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.73099999999999998,"15min":0.71999999999999997},"min":{"1min":0.59099999999999997,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94299999999999995,"5min":1.147,"15min":1.147},"last":0.74199999999999999}]},{"osd":5,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.67400000000000004,"15min":0.69299999999999995},"min":{"1min":0.40300000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78300000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66900000000000004,"15min":0.67800000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84799999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55600000000000005}]},{"osd":6,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.45200000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.76200000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67300000000000004,"15min":0.68700000000000006},"min":{"1min":0.47299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78800000000000003,"5min":1.2,"15min":1.2},"last":0.57499999999999996}]},{"osd":7,"last update":"Sat Aug 17 02:48:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70699999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.626},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.70399999999999996,"15min":0.71399999999999997},"min":{"1min":0.504,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.53600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302052,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069788,"statfs":{"total":96636764160,"available":96327462912,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.48699999999999999,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.627,"5min":1.901,"15min":1.901},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.59699999999999998,"15min":0.63300000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77400000000000002,"5min":3.552,"15min":3.552},"last":0.44800000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.496,"15min":0.50600000000000001},"min":{"1min":0.33200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.67200000000000004,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.57599999999999996,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73499999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.64700000000000002,"15min":0.69799999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73599999999999999,"5min":3.8149999999999999,"15min":3.8149999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.625,"15min":0.63900000000000001},"min":{"1min":0.502,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73399999999999999,"5min":1.974,"15min":1.974},"last":0.55500000000000005}]},{"osd":4,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.85699999999999998,"15min":0.88700000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.266,"5min":3.9009999999999998,"15min":3.9009999999999998},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.89600000000000002,"15min":0.94999999999999996},"min":{"1min":0.56999999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.206,"5min":4.3360000000000003,"15min":4.3360000000000003},"last":0.81499999999999995}]},{"osd":5,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.84699999999999998,"15min":0.874},"min":{"1min":0.60199999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0860000000000001,"5min":3.371,"15min":3.371},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.873,"15min":0.89900000000000002},"min":{"1min":0.59399999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1679999999999999,"5min":3.8599999999999999,"15min":3.8599999999999999},"last":0.96299999999999997}]},{"osd":6,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.91000000000000003,"15min":0.96099999999999997},"min":{"1min":0.58299999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.3109999999999999,"5min":4.4160000000000004,"15min":4.4160000000000004},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.89100000000000001,"15min":0.91700000000000004},"min":{"1min":0.72699999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.2350000000000001,"5min":3.331,"15min":3.331},"last":0.82299999999999995}]},{"osd":7,"last update":"Sat Aug 17 02:48:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.94099999999999995,"15min":0.98299999999999998},"min":{"1min":0.66500000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.105,"5min":3.931,"15min":3.931},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.95699999999999996,"15min":1.0029999999999999},"min":{"1min":0.67600000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.4279999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.77600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302056,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069784,"statfs":{"total":96636764160,"available":96327458816,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.443,"15min":0.45900000000000002},"min":{"1min":0.314,"5min":0.255,"15min":0.255},"max":{"1min":0.58399999999999996,"5min":1.843,"15min":1.843},"last":0.377},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52600000000000002,"15min":0.54800000000000004},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.58599999999999997,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.50800000000000001}]},{"osd":1,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54500000000000004,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.73499999999999999,"5min":1.903,"15min":1.903},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.61499999999999999,"15min":0.66200000000000003},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.60299999999999998,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.56499999999999995}]},{"osd":3,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.72399999999999998,"15min":0.81899999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68799999999999994,"5min":7.5960000000000001,"15min":7.5960000000000001},"last":0.436},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.77000000000000002,"15min":0.83999999999999997},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82299999999999995,"5min":6.0499999999999998,"15min":6.0499999999999998},"last":0.53500000000000003}]},{"osd":4,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.85399999999999998},"min":{"1min":0.72899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.97599999999999998,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":1.0069999999999999,"15min":1.099},"min":{"1min":0.60999999999999999,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.0860000000000001,"5min":5.7009999999999996,"15min":5.7009999999999996},"last":0.83299999999999996}]},{"osd":5,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.96199999999999997,"15min":1.0569999999999999},"min":{"1min":0.56399999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.006,"5min":7.5220000000000002,"15min":7.5220000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.91200000000000003,"15min":0.97099999999999997},"min":{"1min":0.57399999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.91900000000000004,"5min":5.968,"15min":5.968},"last":0.879}]},{"osd":6,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":1.012,"15min":1.1100000000000001},"min":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.99399999999999999,"5min":5.8780000000000001,"15min":5.8780000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.96999999999999997,"15min":1.0600000000000001},"min":{"1min":0.64500000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0009999999999999,"5min":7.5570000000000004,"15min":7.5570000000000004},"last":0.90000000000000002}]},{"osd":7,"last update":"Sat Aug 17 02:48:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":1.03,"15min":1.121},"min":{"1min":0.70699999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.054,"5min":5.9119999999999999,"15min":5.9119999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":1.0189999999999999,"15min":1.113},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.95799999999999996,"5min":7.4800000000000004,"15min":7.4800000000000004},"last":0.84299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302628,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069212,"statfs":{"total":96636764160,"available":96326873088,"internally_reserved":0,"allocated":1142784,"data_stored":822418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.316,"5min":0.255,"15min":0.255},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.57199999999999995,"15min":0.59999999999999998},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.70699999999999996,"5min":2.6059999999999999,"15min":2.6059999999999999},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69199999999999995,"5min":2.165,"15min":2.165},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.40100000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.64500000000000002,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.377}]},{"osd":3,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.47899999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82299999999999995,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.64100000000000001},"min":{"1min":0.41599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.85899999999999999,"5min":2.9399999999999999,"15min":2.9399999999999999},"last":0.48299999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.66400000000000003,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0229999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76900000000000002,"15min":0.75900000000000001},"min":{"1min":0.66000000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.622,"5min":0.501,"15min":0.501},"max":{"1min":1.034,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.68500000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.085,"5min":1.131,"15min":1.131},"last":0.79000000000000004}]},{"osd":6,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.746},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.89900000000000002,"15min":0.92300000000000004},"min":{"1min":0.65000000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.1060000000000001,"5min":2.5059999999999998,"15min":2.5059999999999998},"last":0.70899999999999996}]},{"osd":7,"last update":"Sat Aug 17 02:48:24 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.85599999999999998,"15min":0.872},"min":{"1min":0.67000000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0700000000000001,"5min":2.577,"15min":2.577},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81999999999999995,"15min":0.82099999999999995},"min":{"1min":0.63500000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.026,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302052,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069788,"statfs":{"total":96636764160,"available":96327462912,"internally_reserved":0,"allocated":548864,"data_stored":232050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.317,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.46899999999999997,"15min":0.46100000000000002},"min":{"1min":0.40000000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67400000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.53100000000000003}]},{"osd":2,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53100000000000003,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.70999999999999996,"5min":1.105,"15min":1.105},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.44600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73899999999999999,"5min":0.871,"15min":0.871},"last":0.57299999999999995}]},{"osd":3,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.46100000000000002},"min":{"1min":0.39200000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.754,"5min":1.181,"15min":1.181},"last":0.48299999999999998}]},{"osd":4,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78400000000000003,"15min":0.77900000000000003},"min":{"1min":0.67600000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.093,"5min":1.468,"15min":1.468},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.71899999999999997,"15min":0.70799999999999996},"min":{"1min":0.56499999999999995,"5min":0.44,"15min":0.44},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.70699999999999996}]},{"osd":5,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.59399999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75600000000000001,"15min":0.753},"min":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.91700000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81100000000000005,"15min":0.82099999999999995},"min":{"1min":0.60999999999999999,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96599999999999997,"5min":1.444,"15min":1.444},"last":0.748},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80600000000000005,"15min":0.81499999999999995},"min":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.01,"5min":1.756,"15min":1.756},"last":0.76900000000000002}]},{"osd":7,"last update":"Sat Aug 17 02:48:16 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81899999999999995,"15min":0.81999999999999995},"min":{"1min":0.67800000000000005,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1339999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.65800000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.99299999999999999,"5min":1.298,"15min":1.298},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-17T02:49:17.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-08-17T02:49:17.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.901+0000 7f1a52800640 1 Processor -- start 2024-08-17T02:49:17.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.901+0000 7f1a52800640 1 -- start start 2024-08-17T02:49:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.902+0000 7f1a52800640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f1a4c156c50 0x7f1a4c0c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.902+0000 7f1a52800640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a4c0adcf0 0x7f1a4c0ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.902+0000 7f1a52800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a4c0abb00 0x7f1a4c0c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.902+0000 7f1a52800640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4c0ab130 con 0x7f1a4c0adcf0 2024-08-17T02:49:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.902+0000 7f1a52800640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4c0ae5a0 con 0x7f1a4c156c50 2024-08-17T02:49:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.902+0000 7f1a52800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0ae2c0 con 0x7f1a4c0abb00 2024-08-17T02:49:17.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.902+0000 7f1a4aa00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a4c0adcf0 0x7f1a4c0ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.903+0000 7f1a4aa00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a4c0adcf0 0x7f1a4c0ab5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48594/0 (socket says 172.21.15.16:48594) 2024-08-17T02:49:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.903+0000 7f1a4aa00640 1 -- 172.21.15.16:0/443832676 learned_addr learned my addr 172.21.15.16:0/443832676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:49:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.903+0000 7f1a4b400640 1 --2- 172.21.15.16:0/443832676 >> v2:172.21.15.142:3300/0 conn(0x7f1a4c156c50 0x7f1a4c0c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.903+0000 7f1a4be00640 1 --2- 172.21.15.16:0/443832676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a4c0abb00 0x7f1a4c0c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.903+0000 7f1a4aa00640 1 -- 172.21.15.16:0/443832676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a4c0abb00 msgr2=0x7f1a4c0c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:49:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.903+0000 7f1a4aa00640 1 --2- 172.21.15.16:0/443832676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a4c0abb00 0x7f1a4c0c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.903+0000 7f1a4aa00640 1 -- 172.21.15.16:0/443832676 >> v2:172.21.15.142:3300/0 conn(0x7f1a4c156c50 msgr2=0x7f1a4c0c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:49:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.903+0000 7f1a4aa00640 1 --2- 172.21.15.16:0/443832676 >> v2:172.21.15.142:3300/0 conn(0x7f1a4c156c50 0x7f1a4c0c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.903+0000 7f1a4aa00640 1 -- 172.21.15.16:0/443832676 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c0aaae0 con 0x7f1a4c0adcf0 2024-08-17T02:49:17.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.904+0000 7f1a4aa00640 1 --2- 172.21.15.16:0/443832676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a4c0adcf0 0x7f1a4c0ab5c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c06efd0 tx=0x7f1a380091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855f268ebe3e650a server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:49:17.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.904+0000 7f1a4a000640 1 -- 172.21.15.16:0/443832676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3802acb0 con 0x7f1a4c0adcf0 2024-08-17T02:49:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.904+0000 7f1a4a000640 1 -- 172.21.15.16:0/443832676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3802ae10 con 0x7f1a4c0adcf0 2024-08-17T02:49:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.904+0000 7f1a4a000640 1 -- 172.21.15.16:0/443832676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38032670 con 0x7f1a4c0adcf0 2024-08-17T02:49:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.905+0000 7f1a52800640 1 -- 172.21.15.16:0/443832676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a4c0adcf0 msgr2=0x7f1a4c0ab5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.905+0000 7f1a52800640 1 --2- 172.21.15.16:0/443832676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a4c0adcf0 0x7f1a4c0ab5c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c06efd0 tx=0x7f1a380091d0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.905+0000 7f1a52800640 1 -- 172.21.15.16:0/443832676 shutdown_connections 2024-08-17T02:49:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.905+0000 7f1a52800640 1 --2- 172.21.15.16:0/443832676 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a4c0abb00 0x7f1a4c0c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.905+0000 7f1a52800640 1 --2- 172.21.15.16:0/443832676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a4c0adcf0 0x7f1a4c0ab5c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.905+0000 7f1a52800640 1 --2- 172.21.15.16:0/443832676 >> v2:172.21.15.142:3300/0 conn(0x7f1a4c156c50 0x7f1a4c0c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.905+0000 7f1a52800640 1 -- 172.21.15.16:0/443832676 >> 172.21.15.16:0/443832676 conn(0x7f1a4c067750 msgr2=0x7f1a4c0c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:49:17.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.905+0000 7f1a52800640 1 -- 172.21.15.16:0/443832676 shutdown_connections 2024-08-17T02:49:17.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.905+0000 7f1a52800640 1 -- 172.21.15.16:0/443832676 wait complete. 2024-08-17T02:49:17.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.906+0000 7f1a52800640 1 Processor -- start 2024-08-17T02:49:17.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.906+0000 7f1a52800640 1 -- start start 2024-08-17T02:49:17.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.907+0000 7f1a52800640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a4c0abb00 0x7f1a4c159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:17.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.907+0000 7f1a52800640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a4c0adcf0 0x7f1a4c157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:17.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.907+0000 7f1a52800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4c156c50 0x7f1a4c13f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:17.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.907+0000 7f1a52800640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c141bd0 con 0x7f1a4c0abb00 2024-08-17T02:49:17.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.907+0000 7f1a52800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c1429a0 con 0x7f1a4c156c50 2024-08-17T02:49:17.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.907+0000 7f1a52800640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c13d1b0 con 0x7f1a4c0adcf0 2024-08-17T02:49:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.907+0000 7f1a4b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a4c0abb00 0x7f1a4c159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.908+0000 7f1a4b400640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a4c0abb00 0x7f1a4c159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48602/0 (socket says 172.21.15.16:48602) 2024-08-17T02:49:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.908+0000 7f1a4b400640 1 -- 172.21.15.16:0/3374103297 learned_addr learned my addr 172.21.15.16:0/3374103297 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:49:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.908+0000 7f1a4be00640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4c156c50 0x7f1a4c13f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.908+0000 7f1a4aa00640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a4c0adcf0 0x7f1a4c157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.908+0000 7f1a4b400640 1 -- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a4c0adcf0 msgr2=0x7f1a4c157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.908+0000 7f1a4b400640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a4c0adcf0 0x7f1a4c157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.908+0000 7f1a4b400640 1 -- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4c156c50 msgr2=0x7f1a4c13f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:17.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.908+0000 7f1a4b400640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4c156c50 0x7f1a4c13f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:17.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.908+0000 7f1a4b400640 1 -- 172.21.15.16:0/3374103297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a38009450 con 0x7f1a4c0abb00 2024-08-17T02:49:17.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.908+0000 7f1a4b400640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a4c0abb00 0x7f1a4c159a20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00d890 tx=0x7f1a3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:49:17.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.909+0000 7f1a42a00640 1 -- 172.21.15.16:0/3374103297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00de20 con 0x7f1a4c0abb00 2024-08-17T02:49:17.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.909+0000 7f1a42a00640 1 -- 172.21.15.16:0/3374103297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c0098b0 con 0x7f1a4c0abb00 2024-08-17T02:49:17.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.909+0000 7f1a42a00640 1 -- 172.21.15.16:0/3374103297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c009c50 con 0x7f1a4c0abb00 2024-08-17T02:49:17.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.909+0000 7f1a52800640 1 -- 172.21.15.16:0/3374103297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4c141db0 con 0x7f1a4c0abb00 2024-08-17T02:49:17.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.909+0000 7f1a52800640 1 -- 172.21.15.16:0/3374103297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c142350 con 0x7f1a4c0abb00 2024-08-17T02:49:17.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.910+0000 7f1a52800640 1 -- 172.21.15.16:0/3374103297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4c071050 con 0x7f1a4c0abb00 2024-08-17T02:49:17.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.911+0000 7f1a42a00640 1 -- 172.21.15.16:0/3374103297 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f1a3c014620 con 0x7f1a4c0abb00 2024-08-17T02:49:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.911+0000 7f1a42a00640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1a2c065150 0x7f1a2c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.912+0000 7f1a42a00640 1 -- 172.21.15.16:0/3374103297 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f1a3c082940 con 0x7f1a4c0abb00 2024-08-17T02:49:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.912+0000 7f1a4aa00640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1a2c065150 0x7f1a2c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.913+0000 7f1a4aa00640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1a2c065150 0x7f1a2c067600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a38034000 tx=0x7f1a38032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:49:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:17.916+0000 7f1a42a00640 1 -- 172.21.15.16:0/3374103297 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f1a3c050cb0 con 0x7f1a4c0abb00 2024-08-17T02:49:18.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.096+0000 7f1a52800640 1 -- 172.21.15.16:0/3374103297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f1a4c06a830 con 0x7f1a4c0abb00 2024-08-17T02:49:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.101+0000 7f1a42a00640 1 -- 172.21.15.16:0/3374103297 <== mon.0 v2:172.21.15.16:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1a3c050cb0 con 0x7f1a4c0abb00 2024-08-17T02:49:18.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.101+0000 7f1a42a00640 1 -- 172.21.15.16:0/3374103297 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f1a3c0542d0 con 0x7f1a4c0abb00 2024-08-17T02:49:18.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.103+0000 7f1a52800640 1 -- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1a2c065150 msgr2=0x7f1a2c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.103+0000 7f1a52800640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1a2c065150 0x7f1a2c067600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a38034000 tx=0x7f1a38032000 comp rx=0 tx=0).stop 2024-08-17T02:49:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.103+0000 7f1a52800640 1 -- 172.21.15.16:0/3374103297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a4c0abb00 msgr2=0x7f1a4c159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.103+0000 7f1a52800640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a4c0abb00 0x7f1a4c159a20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00d890 tx=0x7f1a3c008040 comp rx=0 tx=0).stop 2024-08-17T02:49:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.103+0000 7f1a52800640 1 -- 172.21.15.16:0/3374103297 shutdown_connections 2024-08-17T02:49:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.103+0000 7f1a52800640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f1a2c065150 0x7f1a2c067600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.103+0000 7f1a52800640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4c156c50 0x7f1a4c13f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.103+0000 7f1a52800640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f1a4c0adcf0 0x7f1a4c157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.103+0000 7f1a52800640 1 --2- 172.21.15.16:0/3374103297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a4c0abb00 0x7f1a4c159a20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.103+0000 7f1a52800640 1 -- 172.21.15.16:0/3374103297 >> 172.21.15.16:0/3374103297 conn(0x7f1a4c067750 msgr2=0x7f1a4c0afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:49:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.103+0000 7f1a52800640 1 -- 172.21.15.16:0/3374103297 shutdown_connections 2024-08-17T02:49:18.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.103+0000 7f1a52800640 1 -- 172.21.15.16:0/3374103297 wait complete. 2024-08-17T02:49:18.120 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-17T02:49:18.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-08-17T02:49:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.270+0000 7f7944200640 1 Processor -- start 2024-08-17T02:49:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.270+0000 7f7944200640 1 -- start start 2024-08-17T02:49:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.271+0000 7f7944200640 1 --2- >> v2:172.21.15.142:3300/0 conn(0x7f793c155a20 0x7f793c0adf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.271+0000 7f7944200640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f793c155df0 0x7f793c0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.271+0000 7f7944200640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793c156670 con 0x7f793c06d7f0 2024-08-17T02:49:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.271+0000 7f7944200640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c0b0350 con 0x7f793c155df0 2024-08-17T02:49:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.271+0000 7f7944200640 1 -- --> v2:172.21.15.142:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793c06e5e0 con 0x7f793c155a20 2024-08-17T02:49:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.271+0000 7f7940e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f793c155df0 0x7f793c0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.271+0000 7f7940e00640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f793c155df0 0x7f793c0ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48616/0 (socket says 172.21.15.16:48616) 2024-08-17T02:49:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.271+0000 7f7940e00640 1 -- 172.21.15.16:0/797056420 learned_addr learned my addr 172.21.15.16:0/797056420 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:49:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.271+0000 7f7941800640 1 --2- 172.21.15.16:0/797056420 >> v2:172.21.15.142:3300/0 conn(0x7f793c155a20 0x7f793c0adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:18.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.272+0000 7f7940e00640 1 -- 172.21.15.16:0/797056420 >> v2:172.21.15.142:3300/0 conn(0x7f793c155a20 msgr2=0x7f793c0adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-17T02:49:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.272+0000 7f7940e00640 1 --2- 172.21.15.16:0/797056420 >> v2:172.21.15.142:3300/0 conn(0x7f793c155a20 0x7f793c0adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.272+0000 7f7940e00640 1 -- 172.21.15.16:0/797056420 >> v1:172.21.15.16:6789/0 conn(0x7f793c06d7f0 legacy=0x7f793c0ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:18.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.272+0000 7f7940e00640 1 -- 172.21.15.16:0/797056420 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c06e8c0 con 0x7f793c155df0 2024-08-17T02:49:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.273+0000 7f7940e00640 1 --2- 172.21.15.16:0/797056420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c155df0 0x7f793c0ad9b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f793c06bc00 tx=0x7f79280091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b5a5aca84a7256 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:49:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.273+0000 7f793be00640 1 -- 172.21.15.16:0/797056420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792802ae80 con 0x7f793c155df0 2024-08-17T02:49:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.273+0000 7f793be00640 1 -- 172.21.15.16:0/797056420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7928032440 con 0x7f793c155df0 2024-08-17T02:49:18.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.273+0000 7f793be00640 1 -- 172.21.15.16:0/797056420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928032820 con 0x7f793c155df0 2024-08-17T02:49:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.273+0000 7f7944200640 1 -- 172.21.15.16:0/797056420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c155df0 msgr2=0x7f793c0ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.273+0000 7f7944200640 1 --2- 172.21.15.16:0/797056420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c155df0 0x7f793c0ad9b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f793c06bc00 tx=0x7f79280091d0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.274+0000 7f7944200640 1 -- 172.21.15.16:0/797056420 shutdown_connections 2024-08-17T02:49:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.274+0000 7f7944200640 1 --2- 172.21.15.16:0/797056420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c155df0 0x7f793c0ad9b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.274+0000 7f7944200640 1 --2- 172.21.15.16:0/797056420 >> v2:172.21.15.142:3300/0 conn(0x7f793c155a20 0x7f793c0adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.274+0000 7f7944200640 1 -- 172.21.15.16:0/797056420 >> 172.21.15.16:0/797056420 conn(0x7f793c0cb9b0 msgr2=0x7f793c0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:49:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.274+0000 7f7944200640 1 -- 172.21.15.16:0/797056420 shutdown_connections 2024-08-17T02:49:18.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.274+0000 7f7944200640 1 -- 172.21.15.16:0/797056420 wait complete. 2024-08-17T02:49:18.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.275+0000 7f7944200640 1 Processor -- start 2024-08-17T02:49:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.275+0000 7f7944200640 1 -- start start 2024-08-17T02:49:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.276+0000 7f7944200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f793c06d7f0 0x7f793c0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.276+0000 7f7944200640 1 --2- >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f793c155a20 0x7f793c0c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.276+0000 7f7944200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c155df0 0x7f793c0c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:18.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.276+0000 7f7944200640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c169850 con 0x7f793c155df0 2024-08-17T02:49:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.276+0000 7f7944200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793c0c4f50 con 0x7f793c06d7f0 2024-08-17T02:49:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.276+0000 7f7944200640 1 -- --> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793c168f30 con 0x7f793c155a20 2024-08-17T02:49:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.276+0000 7f7942200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c155df0 0x7f793c0c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.276+0000 7f7942200640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c155df0 0x7f793c0c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48620/0 (socket says 172.21.15.16:48620) 2024-08-17T02:49:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.276+0000 7f7942200640 1 -- 172.21.15.16:0/3191788506 learned_addr learned my addr 172.21.15.16:0/3191788506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-08-17T02:49:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.276+0000 7f7941800640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f793c06d7f0 0x7f793c0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.276+0000 7f7940e00640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f793c155a20 0x7f793c0c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.277+0000 7f7942200640 1 -- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f793c155a20 msgr2=0x7f793c0c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.277+0000 7f7942200640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f793c155a20 0x7f793c0c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.277+0000 7f7942200640 1 -- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f793c06d7f0 msgr2=0x7f793c0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.277+0000 7f7942200640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f793c06d7f0 0x7f793c0cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.277+0000 7f7942200640 1 -- 172.21.15.16:0/3191788506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7928009450 con 0x7f793c155df0 2024-08-17T02:49:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.277+0000 7f7942200640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c155df0 0x7f793c0c4a10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f793400c7f0 tx=0x7f79340015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:49:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.277+0000 7f7938c00640 1 -- 172.21.15.16:0/3191788506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793400a640 con 0x7f793c155df0 2024-08-17T02:49:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.277+0000 7f7938c00640 1 -- 172.21.15.16:0/3191788506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f793400a7c0 con 0x7f793c155df0 2024-08-17T02:49:18.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.277+0000 7f7938c00640 1 -- 172.21.15.16:0/3191788506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793400aba0 con 0x7f793c155df0 2024-08-17T02:49:18.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.277+0000 7f7944200640 1 -- 172.21.15.16:0/3191788506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793c16d270 con 0x7f793c155df0 2024-08-17T02:49:18.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.278+0000 7f7944200640 1 -- 172.21.15.16:0/3191788506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793c1694e0 con 0x7f793c155df0 2024-08-17T02:49:18.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.279+0000 7f7944200640 1 -- 172.21.15.16:0/3191788506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793c064db0 con 0x7f793c155df0 2024-08-17T02:49:18.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.279+0000 7f7938c00640 1 -- 172.21.15.16:0/3191788506 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 83420+0+0 (secure 0 0 0) 0x7f7934003030 con 0x7f793c155df0 2024-08-17T02:49:18.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.280+0000 7f7938c00640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f791c065150 0x7f791c067600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-17T02:49:18.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.280+0000 7f7938c00640 1 -- 172.21.15.16:0/3191788506 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f79340825b0 con 0x7f793c155df0 2024-08-17T02:49:18.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.281+0000 7f7941800640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f791c065150 0x7f791c067600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-17T02:49:18.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.281+0000 7f7941800640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f791c065150 0x7f791c067600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7930007ba0 tx=0x7f7930008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-17T02:49:18.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.284+0000 7f7938c00640 1 -- 172.21.15.16:0/3191788506 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175657 (secure 0 0 0) 0x7f79340509a0 con 0x7f793c155df0 2024-08-17T02:49:18.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.502+0000 7f7944200640 1 -- 172.21.15.16:0/3191788506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f793c1586c0 con 0x7f793c155df0 2024-08-17T02:49:18.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.503+0000 7f7938c00640 1 -- 172.21.15.16:0/3191788506 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f793400b0d0 con 0x7f793c155df0 2024-08-17T02:49:18.504 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-08-17T02:49:18.505 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4670,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2889208700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2889208700},{"type":"v1","addr":"172.21.15.142:6837","nonce":2889208700}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":4861,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/4287834697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":4287834697},{"type":"v1","addr":"172.21.15.142:6835","nonce":4287834697}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/1841153868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":1841153868},{"type":"v1","addr":"172.21.15.16:6835","nonce":1841153868}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":5406,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/3325790371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":3325790371},{"type":"v1","addr":"172.21.15.16:6834","nonce":3325790371}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2024-08-17T02:49:18.505 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 26 2024-08-17T02:49:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.505+0000 7f7944200640 1 -- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f791c065150 msgr2=0x7f791c067600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.505+0000 7f7944200640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f791c065150 0x7f791c067600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7930007ba0 tx=0x7f7930008040 comp rx=0 tx=0).stop 2024-08-17T02:49:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.505+0000 7f7944200640 1 -- 172.21.15.16:0/3191788506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c155df0 msgr2=0x7f793c0c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-17T02:49:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.505+0000 7f7944200640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c155df0 0x7f793c0c4a10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f793400c7f0 tx=0x7f79340015f0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.505+0000 7f7944200640 1 -- 172.21.15.16:0/3191788506 shutdown_connections 2024-08-17T02:49:18.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.505+0000 7f7944200640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:6832/37028,v1:172.21.15.142:6833/37028] conn(0x7f791c065150 0x7f791c067600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.505+0000 7f7944200640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f793c155df0 0x7f793c0c4a10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.505+0000 7f7944200640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:3301/0,v1:172.21.15.142:6790/0] conn(0x7f793c155a20 0x7f793c0c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.505+0000 7f7944200640 1 --2- 172.21.15.16:0/3191788506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f793c06d7f0 0x7f793c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-17T02:49:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.505+0000 7f7944200640 1 -- 172.21.15.16:0/3191788506 >> 172.21.15.16:0/3191788506 conn(0x7f793c0cb9b0 msgr2=0x7f793c06c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-17T02:49:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.505+0000 7f7944200640 1 -- 172.21.15.16:0/3191788506 shutdown_connections 2024-08-17T02:49:18.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-08-17T02:49:18.506+0000 7f7944200640 1 -- 172.21.15.16:0/3191788506 wait complete. 2024-08-17T02:49:18.522 DEBUG:tasks.cephfs.filesystem:already dead... 2024-08-17T02:49:18.522 INFO:teuthology.misc:Shutting down mds daemons... 2024-08-17T02:49:18.522 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-08-17T02:49:18.522 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:18.536 INFO:tasks.ceph.mds.a:Stopped 2024-08-17T02:49:18.536 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-08-17T02:49:18.536 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:18.591 INFO:tasks.ceph.mds.c:Stopped 2024-08-17T02:49:18.592 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-08-17T02:49:18.592 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:18.608 INFO:tasks.ceph.mds.b:Stopped 2024-08-17T02:49:18.608 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-08-17T02:49:18.608 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:18.664 INFO:tasks.ceph.mds.d:Stopped 2024-08-17T02:49:18.664 INFO:teuthology.misc:Shutting down osd daemons... 2024-08-17T02:49:18.665 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-08-17T02:49:18.665 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:18.706 INFO:tasks.ceph.osd.0:Stopped 2024-08-17T02:49:18.707 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-08-17T02:49:18.707 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:18.748 INFO:tasks.ceph.osd.1:Stopped 2024-08-17T02:49:18.749 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-08-17T02:49:18.749 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:18.797 INFO:tasks.ceph.osd.2:Stopped 2024-08-17T02:49:18.798 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-08-17T02:49:18.798 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:18.855 INFO:tasks.ceph.osd.3:Stopped 2024-08-17T02:49:18.855 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-08-17T02:49:18.855 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:18.896 INFO:tasks.ceph.osd.4:Stopped 2024-08-17T02:49:18.897 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-08-17T02:49:18.897 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:18.943 INFO:tasks.ceph.osd.5:Stopped 2024-08-17T02:49:18.944 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-08-17T02:49:18.944 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:18.988 INFO:tasks.ceph.osd.6:Stopped 2024-08-17T02:49:18.988 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-08-17T02:49:18.988 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:19.042 INFO:tasks.ceph.osd.7:Stopped 2024-08-17T02:49:19.042 INFO:teuthology.misc:Shutting down mgr daemons... 2024-08-17T02:49:19.042 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-08-17T02:49:19.042 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:19.099 INFO:tasks.ceph.mgr.y:Stopped 2024-08-17T02:49:19.099 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-08-17T02:49:19.099 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:19.160 INFO:tasks.ceph.mgr.x:Stopped 2024-08-17T02:49:19.161 INFO:teuthology.misc:Shutting down mon daemons... 2024-08-17T02:49:19.161 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-08-17T02:49:19.161 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:19.180 INFO:tasks.ceph.mon.a:Stopped 2024-08-17T02:49:19.181 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-08-17T02:49:19.181 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:19.198 INFO:tasks.ceph.mon.b:Stopped 2024-08-17T02:49:19.198 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-08-17T02:49:19.198 INFO:teuthology.orchestra.run:waiting for 300 2024-08-17T02:49:19.256 INFO:tasks.ceph.mon.c:Stopped 2024-08-17T02:49:19.256 INFO:tasks.ceph:Checking cluster log for badness... 2024-08-17T02:49:19.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-08-17T02:49:19.337 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:49:19.338 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-08-17T02:49:19.577 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:49:19.577 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-08-17T02:49:19.691 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:49:19.691 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-08-17T02:49:19.790 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:49:19.790 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-08-17T02:49:19.890 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:49:19.890 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-08-17T02:49:20.050 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:49:20.051 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-08-17T02:49:20.145 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:49:20.145 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-08-17T02:49:20.245 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:49:20.245 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-08-17T02:49:20.362 INFO:tasks.ceph:Archiving mon data... 2024-08-17T02:49:20.363 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288/data/mon.a.tgz 2024-08-17T02:49:20.364 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-08-17T02:49:20.400 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.v9FLvtEfko 2024-08-17T02:49:20.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.v9FLvtEfko 2024-08-17T02:49:20.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:49:20.489 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:49:20.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.v9FLvtEfko 2024-08-17T02:49:20.710 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.v9FLvtEfko is 319KB 2024-08-17T02:49:20.800 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.v9FLvtEfko 2024-08-17T02:49:20.835 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288/data/mon.b.tgz 2024-08-17T02:49:20.835 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-08-17T02:49:20.871 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.Gj5hAWsGip 2024-08-17T02:49:20.871 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Gj5hAWsGip 2024-08-17T02:49:21.080 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.Gj5hAWsGip 2024-08-17T02:49:21.175 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.Gj5hAWsGip is 419KB 2024-08-17T02:49:21.266 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.Gj5hAWsGip 2024-08-17T02:49:21.301 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288/data/mon.c.tgz 2024-08-17T02:49:21.301 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-08-17T02:49:21.376 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.MZ2J4N3XeO 2024-08-17T02:49:21.377 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.MZ2J4N3XeO 2024-08-17T02:49:21.556 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.MZ2J4N3XeO 2024-08-17T02:49:21.650 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.MZ2J4N3XeO is 419KB 2024-08-17T02:49:21.740 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.MZ2J4N3XeO 2024-08-17T02:49:21.775 INFO:tasks.ceph:Cleaning ceph cluster... 2024-08-17T02:49:21.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-17T02:49:21.780 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-08-17T02:49:21.872 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-08-17T02:49:21.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-08-17T02:49:21.877 DEBUG:teuthology.orchestra.run.smithi142:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-08-17T02:49:21.938 INFO:teuthology.orchestra.run.smithi142.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-08-17T02:49:21.939 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-08-17T02:49:21.950 INFO:tasks.ceph:Archiving crash dumps... 2024-08-17T02:49:21.955 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288/remote/smithi016/crash 2024-08-17T02:49:21.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-17T02:49:22.011 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288/remote/smithi142/crash 2024-08-17T02:49:22.012 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-08-17T02:49:22.066 INFO:tasks.ceph:Shutting down logrotate 2024-08-17T02:49:22.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:49:22.071 DEBUG:teuthology.orchestra.run.smithi142:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-08-17T02:49:22.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-08-17T02:49:22.206 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-08-17T02:49:22.261 INFO:tasks.ceph:Compressing logs... 2024-08-17T02:49:22.261 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-17T02:49:22.265 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-17T02:49:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42390.log 2024-08-17T02:49:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42405.log 2024-08-17T02:49:22.317 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48987.log 2024-08-17T02:49:22.318 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42390.log.gz 2024-08-17T02:49:22.318 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37510.log 2024-08-17T02:49:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44623.log 2024-08-17T02:49:22.319 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42405.log.gz 2024-08-17T02:49:22.319 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48464.log 2024-08-17T02:49:22.322 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48987.log.gz 2024-08-17T02:49:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44623.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44623.log.gz 2024-08-17T02:49:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50484.log 2024-08-17T02:49:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48464.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30849.log 2024-08-17T02:49:22.322 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48464.log.gz 2024-08-17T02:49:22.323 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43148.log 2024-08-17T02:49:22.323 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50484.log.gz 2024-08-17T02:49:22.324 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-17T02:49:22.324 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37510.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-17T02:49:22.324 INFO:teuthology.orchestra.run.smithi142.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37510.log.gz 2024-08-17T02:49:22.324 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43658.log 2024-08-17T02:49:22.324 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph.audit.log: 91.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-17T02:49:22.325 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-08-17T02:49:22.325 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.1.43658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43658.log.gz 2024-08-17T02:49:22.325 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-08-17T02:49:22.325 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.7.log: 93.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-17T02:49:22.325 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-08-17T02:49:22.325 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-08-17T02:49:22.329 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37390.log 2024-08-17T02:49:22.329 INFO:teuthology.orchestra.run.smithi142.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-08-17T02:49:22.329 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37219.log 2024-08-17T02:49:22.329 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37390.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37390.log.gz 2024-08-17T02:49:22.329 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-08-17T02:49:22.329 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37219.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-08-17T02:49:22.329 INFO:teuthology.orchestra.run.smithi142.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37219.log.gz 2024-08-17T02:49:22.333 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-08-17T02:49:22.334 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37470.log 2024-08-17T02:49:22.334 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37430.log 2024-08-17T02:49:22.334 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37470.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37470.log.gz 2024-08-17T02:49:22.335 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.tmp-client.admin.30849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48296.log 2024-08-17T02:49:22.335 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30849.log.gz 2024-08-17T02:49:22.336 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42051.log 2024-08-17T02:49:22.336 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43148.log.gz 2024-08-17T02:49:22.336 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50358.log 2024-08-17T02:49:22.336 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48296.log.gz 2024-08-17T02:49:22.336 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45794.log 2024-08-17T02:49:22.336 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42051.log.gz 2024-08-17T02:49:22.336 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50358.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50358.log.gz 2024-08-17T02:49:22.336 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46270.log 2024-08-17T02:49:22.337 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42402.log 2024-08-17T02:49:22.337 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45794.log.gz 2024-08-17T02:49:22.337 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48735.log 2024-08-17T02:49:22.337 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46270.log.gz 2024-08-17T02:49:22.337 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48422.log 2024-08-17T02:49:22.337 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42402.log.gz 2024-08-17T02:49:22.337 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49071.log 2024-08-17T02:49:22.337 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48735.log.gz 2024-08-17T02:49:22.337 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47734.log 2024-08-17T02:49:22.338 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48422.log.gz 2024-08-17T02:49:22.338 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46396.log 2024-08-17T02:49:22.338 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49071.log.gz 2024-08-17T02:49:22.338 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46312.log 2024-08-17T02:49:22.338 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47734.log.gz 2024-08-17T02:49:22.338 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46870.log 2024-08-17T02:49:22.338 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46396.log.gz 2024-08-17T02:49:22.338 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46312.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46312.log.gz 2024-08-17T02:49:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49728.log 2024-08-17T02:49:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45703.log 2024-08-17T02:49:22.339 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46870.log.gz 2024-08-17T02:49:22.339 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49728.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-08-17T02:49:22.339 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49728.log.gz 2024-08-17T02:49:22.344 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37430.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37430.log.gz 2024-08-17T02:49:22.344 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37259.log 2024-08-17T02:49:22.344 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42930.log 2024-08-17T02:49:22.345 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37259.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37259.log.gz 2024-08-17T02:49:22.345 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.1.42930.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42930.log.gz 2024-08-17T02:49:22.345 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43251.log 2024-08-17T02:49:22.345 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37350.log 2024-08-17T02:49:22.345 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.3.43251.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43251.log.gz 2024-08-17T02:49:22.345 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37350.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37350.log.gz 2024-08-17T02:49:22.345 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-08-17T02:49:22.349 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-08-17T02:49:22.350 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-08-17T02:49:22.350 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37310.log 2024-08-17T02:49:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48506.log 2024-08-17T02:49:22.351 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45703.log.gz 2024-08-17T02:49:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46522.log 2024-08-17T02:49:22.351 INFO:teuthology.orchestra.run.smithi016.stderr: 84.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-08-17T02:49:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47830.log 2024-08-17T02:49:22.351 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48506.log.gz 2024-08-17T02:49:22.351 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43526.log 2024-08-17T02:49:22.352 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46522.log.gz 2024-08-17T02:49:22.352 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46065.log 2024-08-17T02:49:22.352 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47830.log.gz 2024-08-17T02:49:22.352 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45397.log 2024-08-17T02:49:22.352 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43526.log.gz 2024-08-17T02:49:22.352 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46065.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-08-17T02:49:22.352 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46065.log.gz 2024-08-17T02:49:22.352 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45397.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45397.log.gz 2024-08-17T02:49:22.352 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46776.log 2024-08-17T02:49:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42738.log 2024-08-17T02:49:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49770.log 2024-08-17T02:49:22.353 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46776.log.gz 2024-08-17T02:49:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49384.log 2024-08-17T02:49:22.353 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42738.log.gz 2024-08-17T02:49:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49854.log 2024-08-17T02:49:22.353 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49770.log.gz 2024-08-17T02:49:22.353 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41161.log 2024-08-17T02:49:22.354 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49384.log.gz 2024-08-17T02:49:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43358.log 2024-08-17T02:49:22.354 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49854.log.gz 2024-08-17T02:49:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46648.log 2024-08-17T02:49:22.354 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41161.log.gz 2024-08-17T02:49:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43358.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43358.log.gz 2024-08-17T02:49:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50526.log 2024-08-17T02:49:22.354 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47184.log 2024-08-17T02:49:22.355 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46648.log.gz 2024-08-17T02:49:22.355 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.37310.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37310.log.gz 2024-08-17T02:49:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50400.log 2024-08-17T02:49:22.358 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50526.log.gz 2024-08-17T02:49:22.358 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50566.log 2024-08-17T02:49:22.358 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47184.log.gz 2024-08-17T02:49:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50400.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-08-17T02:49:22.359 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50400.log.gz 2024-08-17T02:49:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45439.log 2024-08-17T02:49:22.359 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50566.log.gz 2024-08-17T02:49:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43106.log 2024-08-17T02:49:22.359 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44959.log 2024-08-17T02:49:22.359 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45439.log.gz 2024-08-17T02:49:22.359 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-08-17T02:49:22.360 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49673.log 2024-08-17T02:49:22.360 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43106.log.gz 2024-08-17T02:49:22.360 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44959.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44959.log.gz 2024-08-17T02:49:22.361 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50106.log 2024-08-17T02:49:22.361 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49342.log 2024-08-17T02:49:22.364 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49673.log.gz 2024-08-17T02:49:22.364 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43232.log 2024-08-17T02:49:22.364 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50106.log.gz 2024-08-17T02:49:22.364 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47776.log 2024-08-17T02:49:22.364 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49342.log.gz 2024-08-17T02:49:22.365 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43232.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43232.log.gz 2024-08-17T02:49:22.365 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50064.log 2024-08-17T02:49:22.365 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44749.log 2024-08-17T02:49:22.365 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47776.log.gz 2024-08-17T02:49:22.365 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48380.log 2024-08-17T02:49:22.366 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50064.log.gz 2024-08-17T02:49:22.366 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42399.log 2024-08-17T02:49:22.366 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44749.log.gz 2024-08-17T02:49:22.367 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48380.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48380.log.gz 2024-08-17T02:49:22.367 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40976.log 2024-08-17T02:49:22.368 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41636.log 2024-08-17T02:49:22.368 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42399.log.gz 2024-08-17T02:49:22.368 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40976.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-08-17T02:49:22.368 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40976.log.gz 2024-08-17T02:49:22.368 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43064.log 2024-08-17T02:49:22.368 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41636.log.gz 2024-08-17T02:49:22.369 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46564.log 2024-08-17T02:49:22.369 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42742.log 2024-08-17T02:49:22.369 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43064.log.gz 2024-08-17T02:49:22.370 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41398.log 2024-08-17T02:49:22.371 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46564.log.gz 2024-08-17T02:49:22.371 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42742.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42742.log.gz 2024-08-17T02:49:22.371 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45573.log 2024-08-17T02:49:22.372 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41678.log 2024-08-17T02:49:22.373 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41398.log.gz 2024-08-17T02:49:22.373 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45573.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45573.log.gz 2024-08-17T02:49:22.373 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48590.log 2024-08-17T02:49:22.374 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49029.log 2024-08-17T02:49:22.375 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41678.log.gz 2024-08-17T02:49:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48590.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48590.log.gz 2024-08-17T02:49:22.376 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40880.log 2024-08-17T02:49:22.377 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49626.log 2024-08-17T02:49:22.377 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49029.log.gz 2024-08-17T02:49:22.378 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.40880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48777.log 2024-08-17T02:49:22.378 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40880.log.gz 2024-08-17T02:49:22.379 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46732.log 2024-08-17T02:49:22.379 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49626.log.gz 2024-08-17T02:49:22.380 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47310.log 2024-08-17T02:49:22.381 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48777.log.gz 2024-08-17T02:49:22.381 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46732.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46732.log.gz 2024-08-17T02:49:22.381 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-08-17T02:49:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46480.log 2024-08-17T02:49:22.383 INFO:teuthology.orchestra.run.smithi016.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47310.log.gz 2024-08-17T02:49:22.383 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42980.log 2024-08-17T02:49:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46480.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46480.log.gz 2024-08-17T02:49:22.384 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45001.log 2024-08-17T02:49:22.386 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42980.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42980.log.gz 2024-08-17T02:49:22.386 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37301.log 2024-08-17T02:49:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49584.log 2024-08-17T02:49:22.387 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45001.log.gz 2024-08-17T02:49:22.388 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37301.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37301.log.gz 2024-08-17T02:49:22.388 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46954.log 2024-08-17T02:49:22.389 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43706.log 2024-08-17T02:49:22.389 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49584.log.gz 2024-08-17T02:49:22.391 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46954.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46954.log.gz 2024-08-17T02:49:22.391 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45355.log 2024-08-17T02:49:22.392 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43706.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-08-17T02:49:22.392 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45355.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43706.log.gz 2024-08-17T02:49:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-08-17T02:49:22.393 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45355.log.gz 2024-08-17T02:49:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44111.log 2024-08-17T02:49:22.395 INFO:teuthology.orchestra.run.smithi142.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-08-17T02:49:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42058.log 2024-08-17T02:49:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42066.log 2024-08-17T02:49:22.397 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44111.log.gz 2024-08-17T02:49:22.398 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44833.log 2024-08-17T02:49:22.398 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42058.log.gz 2024-08-17T02:49:22.399 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42066.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42066.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50190.log 2024-08-17T02:49:22.400 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-08-17T02:49:22.401 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44833.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49426.log 2024-08-17T02:49:22.401 INFO:teuthology.orchestra.run.smithi016.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-08-17T02:49:22.401 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44833.log.gz 2024-08-17T02:49:22.402 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50190.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-08-17T02:49:22.402 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50190.log.gz 2024-08-17T02:49:22.403 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46005.log 2024-08-17T02:49:22.403 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49426.log.gz 2024-08-17T02:49:22.404 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49113.log 2024-08-17T02:49:22.406 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44665.log 2024-08-17T02:49:22.406 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46005.log.gz 2024-08-17T02:49:22.407 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49113.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49113.log.gz 2024-08-17T02:49:22.408 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44047.log 2024-08-17T02:49:22.409 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44665.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44665.log.gz -5 2024-08-17T02:49:22.409 INFO:teuthology.orchestra.run.smithi016.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45043.log 2024-08-17T02:49:22.410 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.44047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48548.log 2024-08-17T02:49:22.410 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44047.log.gz 2024-08-17T02:49:22.411 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48254.log 2024-08-17T02:49:22.411 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45043.log.gz 2024-08-17T02:49:22.413 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46998.log 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48548.log.gz 2024-08-17T02:49:22.413 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-08-17T02:49:22.414 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48254.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46150.log 2024-08-17T02:49:22.414 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48254.log.gz 2024-08-17T02:49:22.415 INFO:teuthology.orchestra.run.smithi016.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-08-17T02:49:22.415 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41888.log 2024-08-17T02:49:22.416 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46998.log.gz 2024-08-17T02:49:22.416 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47608.log 2024-08-17T02:49:22.417 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46150.log.gz 2024-08-17T02:49:22.417 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41888.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41888.log.gz 2024-08-17T02:49:22.417 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41102.log 2024-08-17T02:49:22.418 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43568.log 2024-08-17T02:49:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47608.log: /var/log/ceph/ceph-client.admin.41102.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47608.log.gz 2024-08-17T02:49:22.420 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45313.log 2024-08-17T02:49:22.420 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41102.log.gz 2024-08-17T02:49:22.421 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37196.log 2024-08-17T02:49:22.422 INFO:teuthology.orchestra.run.smithi016.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.43568.log.gz 2024-08-17T02:49:22.422 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47692.log 2024-08-17T02:49:22.422 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45313.log.gz 2024-08-17T02:49:22.423 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.37196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42741.log 2024-08-17T02:49:22.424 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37196.log.gz 2024-08-17T02:49:22.424 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42667.log 2024-08-17T02:49:22.425 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47692.log.gz 2024-08-17T02:49:22.425 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42741.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.42741.log.gz 2024-08-17T02:49:22.426 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43442.log 2024-08-17T02:49:22.426 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44497.log 2024-08-17T02:49:22.427 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42667.log.gz 2024-08-17T02:49:22.428 INFO:teuthology.orchestra.run.smithi142.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-08-17T02:49:22.428 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49896.log 2024-08-17T02:49:22.429 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43442.log.gz 2024-08-17T02:49:22.429 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41310.log 2024-08-17T02:49:22.429 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44497.log.gz 2024-08-17T02:49:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49896.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49896.log.gz 2024-08-17T02:49:22.430 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-08-17T02:49:22.431 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50274.log 2024-08-17T02:49:22.432 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41310.log.gz 2024-08-17T02:49:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48861.log 2024-08-17T02:49:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48212.log 2024-08-17T02:49:22.434 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50274.log.gz 2024-08-17T02:49:22.435 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48819.log 2024-08-17T02:49:22.435 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48861.log.gz 2024-08-17T02:49:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48212.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45271.log 2024-08-17T02:49:22.437 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48212.log.gz 2024-08-17T02:49:22.438 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48819.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47376.log 2024-08-17T02:49:22.438 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48819.log.gz 2024-08-17T02:49:22.439 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48945.log 2024-08-17T02:49:22.439 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45271.log.gz 2024-08-17T02:49:22.441 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46108.log 2024-08-17T02:49:22.441 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47376.log.gz 2024-08-17T02:49:22.442 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44917.log 2024-08-17T02:49:22.442 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48945.log.gz 2024-08-17T02:49:22.443 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41057.log 2024-08-17T02:49:22.444 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46108.log.gz 2024-08-17T02:49:22.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46690.log 2024-08-17T02:49:22.445 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44917.log.gz 2024-08-17T02:49:22.446 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47872.log 2024-08-17T02:49:22.446 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41057.log.gz 2024-08-17T02:49:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46690.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46690.log.gz 2024-08-17T02:49:22.448 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46354.log 2024-08-17T02:49:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47872.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47872.log.gz 2024-08-17T02:49:22.449 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48632.log 2024-08-17T02:49:22.451 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41804.log 2024-08-17T02:49:22.451 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46354.log.gz 2024-08-17T02:49:22.452 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48632.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48632.log.gz 2024-08-17T02:49:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41804.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41804.log.gz 2024-08-17T02:49:22.453 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44411.log 2024-08-17T02:49:22.455 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47958.log 2024-08-17T02:49:22.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.2.44411.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44455.log 2024-08-17T02:49:22.456 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.2.44411.log.gz 2024-08-17T02:49:22.457 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47958.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47958.log.gz 2024-08-17T02:49:22.458 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44455.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44455.log.gz 2024-08-17T02:49:22.459 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43400.log 2024-08-17T02:49:22.460 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47268.log 2024-08-17T02:49:22.461 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42406.log 2024-08-17T02:49:22.462 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43400.log.gz 2024-08-17T02:49:22.463 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45750.log 2024-08-17T02:49:22.463 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47268.log.gz 2024-08-17T02:49:22.464 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42406.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42406.log.gz 2024-08-17T02:49:22.465 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47650.log 2024-08-17T02:49:22.465 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45750.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45750.log.gz 2024-08-17T02:49:22.466 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42057.log 2024-08-17T02:49:22.467 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50232.log 2024-08-17T02:49:22.468 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47650.log.gz 2024-08-17T02:49:22.468 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42057.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42057.log.gz 2024-08-17T02:49:22.470 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50232.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50232.log.gz 2024-08-17T02:49:22.471 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41260.log 2024-08-17T02:49:22.471 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49812.log 2024-08-17T02:49:22.473 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43316.log 2024-08-17T02:49:22.473 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41260.log.gz 2024-08-17T02:49:22.475 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49812.log: gzip -5 --verbose 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49812.log.gz 2024-08-17T02:49:22.475 INFO:teuthology.orchestra.run.smithi016.stderr: -- /var/log/ceph/ceph-client.admin.50442.log 2024-08-17T02:49:22.475 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43316.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43316.log.gz 2024-08-17T02:49:22.476 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45837.log 2024-08-17T02:49:22.477 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42070.log 2024-08-17T02:49:22.477 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50442.log.gz 2024-08-17T02:49:22.479 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50608.log 2024-08-17T02:49:22.479 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45837.log.gz 2024-08-17T02:49:22.480 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36916.log 2024-08-17T02:49:22.480 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42070.log.gz 2024-08-17T02:49:22.481 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42657.log 2024-08-17T02:49:22.481 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50608.log.gz 2024-08-17T02:49:22.483 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.36916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42737.log 2024-08-17T02:49:22.483 INFO:teuthology.orchestra.run.smithi016.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.36916.log.gz 2024-08-17T02:49:22.484 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42657.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48692.log 2024-08-17T02:49:22.484 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42657.log.gz 2024-08-17T02:49:22.485 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42737.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42737.log.gz 2024-08-17T02:49:22.486 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47914.log 2024-08-17T02:49:22.487 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48692.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48692.log.gz 2024-08-17T02:49:22.487 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50022.log 2024-08-17T02:49:22.488 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47563.log 2024-08-17T02:49:22.489 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47914.log.gz 2024-08-17T02:49:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50022.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50022.log.gz 2024-08-17T02:49:22.491 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.0.47563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47563.log.gz 2024-08-17T02:49:22.491 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47100.log 2024-08-17T02:49:22.492 INFO:teuthology.orchestra.run.smithi142.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-08-17T02:49:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44791.log 2024-08-17T02:49:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41466.log 2024-08-17T02:49:22.494 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47100.log.gz 2024-08-17T02:49:22.495 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49938.log 2024-08-17T02:49:22.495 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44791.log.gz 2024-08-17T02:49:22.497 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42391.log 2024-08-17T02:49:22.497 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41466.log.gz 2024-08-17T02:49:22.498 INFO:teuthology.orchestra.run.smithi142.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-08-17T02:49:22.498 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41846.log 2024-08-17T02:49:22.498 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49938.log.gz 2024-08-17T02:49:22.499 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45229.log 2024-08-17T02:49:22.499 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42391.log.gz 2024-08-17T02:49:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41846.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41846.log.gz 2024-08-17T02:49:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.45229.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44875.log 2024-08-17T02:49:22.502 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45229.log.gz 2024-08-17T02:49:22.502 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45615.log 2024-08-17T02:49:22.504 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48903.log 2024-08-17T02:49:22.504 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44875.log.gz 2024-08-17T02:49:22.505 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45615.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45615.log.gz 2024-08-17T02:49:22.506 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45879.log 2024-08-17T02:49:22.506 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48903.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48903.log.gz 2024-08-17T02:49:22.507 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42396.log 2024-08-17T02:49:22.508 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50148.log 2024-08-17T02:49:22.509 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45879.log.gz 2024-08-17T02:49:22.510 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49300.log 2024-08-17T02:49:22.510 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42396.log.gz 2024-08-17T02:49:22.511 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48128.log 2024-08-17T02:49:22.511 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50148.log.gz 2024-08-17T02:49:22.512 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46606.log 2024-08-17T02:49:22.512 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49300.log.gz 2024-08-17T02:49:22.513 INFO:teuthology.orchestra.run.smithi142.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-08-17T02:49:22.514 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48128.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.41550.log 2024-08-17T02:49:22.514 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48128.log.gz 2024-08-17T02:49:22.515 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47142.log 2024-08-17T02:49:22.515 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46606.log.gz 2024-08-17T02:49:22.516 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46438.log 2024-08-17T02:49:22.516 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41550.log.gz 2024-08-17T02:49:22.518 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41218.log 2024-08-17T02:49:22.518 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47142.log.gz 2024-08-17T02:49:22.519 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46438.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46438.log.gz 2024-08-17T02:49:22.519 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49197.log 2024-08-17T02:49:22.520 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41218.log: gzip -5 --verbose 87.8% -- /var/log/ceph/ceph-client.admin.42403.log -- replaced with /var/log/ceph/ceph-client.admin.41218.log.gz 2024-08-17T02:49:22.521 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-08-17T02:49:22.522 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41594.log 2024-08-17T02:49:22.522 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49197.log.gz 2024-08-17T02:49:22.523 INFO:teuthology.orchestra.run.smithi142.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-08-17T02:49:22.524 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42403.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42403.log.gz 2024-08-17T02:49:22.524 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42740.log 2024-08-17T02:49:22.526 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41594.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41594.log.gz 2024-08-17T02:49:22.526 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.42740.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45963.log 2024-08-17T02:49:22.526 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42740.log.gz 2024-08-17T02:49:22.527 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49155.log 2024-08-17T02:49:22.528 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45963.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45963.log.gz 2024-08-17T02:49:22.529 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-08-17T02:49:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44581.log 2024-08-17T02:49:22.531 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49155.log.gz 2024-08-17T02:49:22.532 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47045.log 2024-08-17T02:49:22.532 INFO:teuthology.orchestra.run.smithi016.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-08-17T02:49:22.533 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43022.log 2024-08-17T02:49:22.533 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44581.log.gz 2024-08-17T02:49:22.534 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47045.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47045.log.gz 2024-08-17T02:49:22.535 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43190.log 2024-08-17T02:49:22.536 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43022.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43022.log.gz 2024-08-17T02:49:22.536 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49980.log 2024-08-17T02:49:22.537 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45658.log 2024-08-17T02:49:22.538 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43190.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gz 2024-08-17T02:49:22.538 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49980.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49980.log.gz 2024-08-17T02:49:22.539 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49257.log 2024-08-17T02:49:22.540 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45658.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45658.log.gz 2024-08-17T02:49:22.540 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45103.log 2024-08-17T02:49:22.541 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46912.log 2024-08-17T02:49:22.542 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.49257.log: 88.4%/var/log/ceph/ceph-client.admin.45103.log: -- replaced with /var/log/ceph/ceph-client.admin.49257.log.gz 2024-08-17T02:49:22.542 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45103.log.gz 2024-08-17T02:49:22.543 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50316.log 2024-08-17T02:49:22.544 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46912.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46912.log.gz 2024-08-17T02:49:22.544 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45921.log 2024-08-17T02:49:22.545 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45187.log 2024-08-17T02:49:22.546 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.50316.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50316.log.gz 2024-08-17T02:49:22.547 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45921.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45921.log.gz 2024-08-17T02:49:22.547 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43274.log 2024-08-17T02:49:22.548 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45187.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45187.log.gz 2024-08-17T02:49:22.548 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42938.log 2024-08-17T02:49:22.549 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43274.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43274.log.gz 2024-08-17T02:49:22.550 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44707.log 2024-08-17T02:49:22.551 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41354.log 2024-08-17T02:49:22.551 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42938.log.gz 2024-08-17T02:49:22.552 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44707.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44707.log.gz 2024-08-17T02:49:22.553 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41720.log 2024-08-17T02:49:22.553 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41354.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41354.log.gz 2024-08-17T02:49:22.554 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42062.log 2024-08-17T02:49:22.555 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41762.log 2024-08-17T02:49:22.556 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41720.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41720.log.gz 2024-08-17T02:49:22.556 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42062.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42062.log.gz 2024-08-17T02:49:22.557 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48338.log 2024-08-17T02:49:22.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.41762.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41762.log.gz 2024-08-17T02:49:22.558 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42665.log 2024-08-17T02:49:22.560 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42069.log 2024-08-17T02:49:22.560 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48338.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48338.log.gz 2024-08-17T02:49:22.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42665.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42665.log.gz 2024-08-17T02:49:22.561 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45145.log 2024-08-17T02:49:22.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.42069.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42069.log.gz 2024-08-17T02:49:22.567 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42063.log 2024-08-17T02:49:22.567 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.45145.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45145.log.gz 2024-08-17T02:49:22.567 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44539.log 2024-08-17T02:49:22.567 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.42063.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46820.log 2024-08-17T02:49:22.567 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42063.log.gz 2024-08-17T02:49:22.568 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.44539.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44539.log.gz 2024-08-17T02:49:22.568 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48170.log 2024-08-17T02:49:22.568 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.46820.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46820.log.gz 2024-08-17T02:49:22.569 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41508.log 2024-08-17T02:49:22.569 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.48170.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48170.log.gz 2024-08-17T02:49:22.570 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43484.log 2024-08-17T02:49:22.571 INFO:teuthology.orchestra.run.smithi016.stderr:gzip/var/log/ceph/ceph-client.admin.41508.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47226.log 2024-08-17T02:49:22.571 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41508.log.gz 2024-08-17T02:49:22.572 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.43484.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43484.log.gz 2024-08-17T02:49:22.573 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.47226.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47226.log.gz 2024-08-17T02:49:22.576 INFO:teuthology.orchestra.run.smithi142.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-08-17T02:49:22.578 INFO:teuthology.orchestra.run.smithi016.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-08-17T02:49:22.579 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-08-17T02:49:22.580 INFO:teuthology.orchestra.run.smithi142.stderr:real 0m0.282s 2024-08-17T02:49:22.580 INFO:teuthology.orchestra.run.smithi142.stderr:user 0m1.047s 2024-08-17T02:49:22.580 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m0.170s 2024-08-17T02:49:22.602 INFO:teuthology.orchestra.run.smithi016.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-08-17T02:49:22.625 INFO:teuthology.orchestra.run.smithi016.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-08-17T02:49:22.643 INFO:teuthology.orchestra.run.smithi016.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-08-17T02:49:22.692 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-08-17T02:49:22.695 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-08-17T02:49:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m0.401s 2024-08-17T02:49:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m1.230s 2024-08-17T02:49:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m0.629s 2024-08-17T02:49:22.696 INFO:tasks.ceph:Archiving logs... 2024-08-17T02:49:22.697 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288/remote/smithi016/log 2024-08-17T02:49:22.698 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-17T02:49:23.064 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288/remote/smithi142/log 2024-08-17T02:49:23.065 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-17T02:49:23.258 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-17T02:49:23.268 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-17T02:49:23.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-17T02:49:23.272 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-17T02:49:23.402 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-17T02:49:23.411 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-17T02:49:23.412 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-17T02:49:23.416 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-17T02:49:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-08-17T02:49:23.449 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-08-17T02:49:23.452 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-17T02:49:23.453 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-08-17T02:49:23.453 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 -9236ns[ -39ns] +/- 45ms 2024-08-17T02:49:23.453 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 -65us[ -56us] +/- 79ms 2024-08-17T02:49:23.453 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +83us[ +92us] +/- 64ms 2024-08-17T02:49:23.453 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-08-17T02:49:23.456 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-17T02:49:23.456 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-08-17T02:49:23.456 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 +22us[ +47us] +/- 45ms 2024-08-17T02:49:23.456 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 -5949ns[-5949ns] +/- 79ms 2024-08-17T02:49:23.456 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +111us[ +111us] +/- 64ms 2024-08-17T02:49:23.457 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-08-17T02:49:23.457 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-17T02:49:23.469 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-17T02:49:23.470 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-17T02:49:23.478 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-17T02:49:23.616 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-17T02:49:23.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-08-17T02:49:23.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:49:23.789 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-08-17T02:49:23.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:49:23.849 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-17T02:49:23.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-17T02:49:23.867 INFO:teuthology.task.internal:Duration was 748.461191 seconds 2024-08-17T02:49:23.868 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-17T02:49:23.876 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-17T02:49:23.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-17T02:49:23.880 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-17T02:49:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-17T02:49:23.967 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-17T02:49:24.401 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-17T02:49:24.401 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-08-17T02:49:24.402 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-17T02:49:24.452 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-08-17T02:49:24.453 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-17T02:49:24.501 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-17T02:49:24.501 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-17T02:49:24.505 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-17T02:49:24.565 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-17T02:49:24.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-17T02:49:24.569 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-17T02:49:24.713 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-17T02:49:24.725 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-17T02:49:24.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-17T02:49:24.782 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-17T02:49:24.838 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-17T02:49:24.849 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-17T02:49:24.853 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-17T02:49:24.920 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-08-17T02:49:24.923 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-08-17T02:49:24.953 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-17T02:49:24.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:49:24.987 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-17T02:49:25.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-17T02:49:25.022 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-17T02:49:25.032 INFO:teuthology.task.internal:Transferring archived files... 2024-08-17T02:49:25.033 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288/remote/smithi016 2024-08-17T02:49:25.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-17T02:49:25.096 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-08-17_01:10:27-fs-wip-mchangir-mgr-volumes-print-thread-name-quincy-distro-default-smithi/7858288/remote/smithi142 2024-08-17T02:49:25.097 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-17T02:49:25.160 INFO:teuthology.task.internal:Removing archive directory... 2024-08-17T02:49:25.161 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-17T02:49:25.165 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-17T02:49:25.200 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-17T02:49:25.215 INFO:teuthology.task.internal:Not uploading archives. 2024-08-17T02:49:25.215 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-17T02:49:25.227 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-17T02:49:25.227 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-17T02:49:25.242 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-17T02:49:25.273 INFO:teuthology.orchestra.run.smithi016.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 17 02:49 /home/ubuntu/cephtest 2024-08-17T02:49:25.276 INFO:teuthology.orchestra.run.smithi142.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 17 02:49 /home/ubuntu/cephtest 2024-08-17T02:49:25.279 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-17T02:49:25.289 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-17T02:49:25.313 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 client.1-kernel-sha1: 7503b862ff5bc4e556b3daee1c7881aa3e90ed62 description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/basic}} duration: 748.4611914157867 failure_reason: 'Test failure: test_volume_create (tasks.cephfs.test_volumes.TestVolumes)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=1b4aff9bde9c42f09eba9cae6de3ac52 status: fail success: false 2024-08-17T02:49:25.314 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-17T02:49:25.394 INFO:teuthology.run:FAIL